2024-09-15T21:21:09.085 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T21:21:09.103 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:21:09.194 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715 branch: main description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-comp-zlib} rados tasks/mon_recovery validater/lockdep} email: ceph-qa@ceph.com first_in_suite: false job_id: '7905715' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: lockdep: true mon client directed command retry: 5 mon election default strategy: 1 ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluestore block size: 96636764160 bluestore compression algorithm: zlib bluestore compression mode: aggressive bluestore fsck on mount: true bluestore write v2: false bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(OSDMAP_FLAGS\) - \(SMALLER_PGP_NUM\) - \(POOL_APP_NOT_ENABLED\) - \(SLOW OPS\) - slow request sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 7498493279982dfe87f00616198a5967475f6169 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 2855 sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 sleep_before_teardown: 0 subset: 17936/100000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7498493279982dfe87f00616198a5967475f6169 targets: smithi019.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKrAKUib0qGfdgL8H4yxZJEGEgYd319GFBz4DpVOZ5PlazZ2iS7ZDGOH+JUqC1l8J97fIgL/I8Ssvo3npqmbKfI= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEocJVGefvwGCDH9LbgyByRr4qAT7uWQEUjUWwN+GvY9pHrDJBkHeNbjJ4ZnjDvvda641wwAI77RzGjrnkPm1q8= tasks: - install: null - ceph: null - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - mon_recovery: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_20:00:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T21:21:09.195 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa; will attempt to use it 2024-09-15T21:21:09.195 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa/tasks 2024-09-15T21:21:09.196 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T21:21:09.197 INFO:teuthology.task.internal:Checking packages... 2024-09-15T21:21:09.219 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'fc3e0bff7468fd55478adbae7f75508b749124b6' 2024-09-15T21:21:09.220 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:21:09.220 INFO:teuthology.packaging:ref: None 2024-09-15T21:21:09.220 INFO:teuthology.packaging:tag: None 2024-09-15T21:21:09.220 INFO:teuthology.packaging:branch: main 2024-09-15T21:21:09.220 INFO:teuthology.packaging:sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:21:09.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-09-15T21:21:09.470 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4930.gfc3e0bff 2024-09-15T21:21:09.472 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T21:21:09.478 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T21:21:09.478 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T21:21:09.528 INFO:teuthology.task.internal:Saving configuration 2024-09-15T21:21:09.542 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T21:21:09.549 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T21:21:09.574 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi019.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:16:10.707484', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKrAKUib0qGfdgL8H4yxZJEGEgYd319GFBz4DpVOZ5PlazZ2iS7ZDGOH+JUqC1l8J97fIgL/I8Ssvo3npqmbKfI='} 2024-09-15T21:21:09.600 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:16:10.706277', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEocJVGefvwGCDH9LbgyByRr4qAT7uWQEUjUWwN+GvY9pHrDJBkHeNbjJ4ZnjDvvda641wwAI77RzGjrnkPm1q8='} 2024-09-15T21:21:09.600 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T21:21:09.608 INFO:teuthology.task.internal:roles: ubuntu@smithi019.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-15T21:21:09.608 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-15T21:21:09.608 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T21:21:09.673 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f721f7dbac0>, signals=[15]) 2024-09-15T21:21:09.673 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T21:21:09.681 INFO:teuthology.task.internal:Opening connections... 2024-09-15T21:21:09.681 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:21:09.682 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:21:09.771 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:21:09.772 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:21:09.854 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T21:21:09.861 DEBUG:teuthology.orchestra.run.smithi019:> uname -m 2024-09-15T21:21:09.879 INFO:teuthology.orchestra.run.smithi019.stdout:x86_64 2024-09-15T21:21:09.880 DEBUG:teuthology.orchestra.run.smithi019:> cat /etc/os-release 2024-09-15T21:21:09.937 INFO:teuthology.orchestra.run.smithi019.stdout:NAME="CentOS Stream" 2024-09-15T21:21:09.937 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION="9" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:ID="centos" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:VERSION_ID="9" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:ANSI_COLOR="0;31" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:21:09.938 INFO:teuthology.orchestra.run.smithi019.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:21:09.939 INFO:teuthology.lock.ops:Updating smithi019.front.sepia.ceph.com on lock server 2024-09-15T21:21:09.960 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2024-09-15T21:21:09.980 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2024-09-15T21:21:09.980 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2024-09-15T21:21:10.038 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2024-09-15T21:21:10.038 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2024-09-15T21:21:10.038 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2024-09-15T21:21:10.038 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:21:10.039 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:21:10.040 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2024-09-15T21:21:10.060 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T21:21:10.070 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T21:21:10.077 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T21:21:10.077 DEBUG:teuthology.orchestra.run.smithi019:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:21:10.080 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:21:10.097 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T21:21:10.104 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T21:21:10.104 DEBUG:teuthology.orchestra.run.smithi019:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:21:10.137 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:21:10.167 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T21:21:10.222 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T21:21:10.239 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T21:21:10.239 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T21:21:10.240 DEBUG:teuthology.orchestra.run.smithi019:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:21:10.240 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:21:10.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:10.256 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-09-15T21:21:10.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:10.257 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-09-15T21:21:10.313 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-503.el9.x86_64 2024-09-15T21:21:10.314 INFO:teuthology.task.kernel:Running kernel on smithi019: 5.14.0-503.el9.x86_64 2024-09-15T21:21:10.314 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum install -y kernel 2024-09-15T21:21:10.315 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-503.el9.x86_64 2024-09-15T21:21:10.315 INFO:teuthology.task.kernel:Running kernel on smithi192: 5.14.0-503.el9.x86_64 2024-09-15T21:21:10.315 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum install -y kernel 2024-09-15T21:21:12.534 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-15T21:21:12.711 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-15T21:21:15.351 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-15T21:21:16.170 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 8.7 MB/s | 20 MB 00:02 2024-09-15T21:21:20.429 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-15T21:21:21.077 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-15T21:21:21.855 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 85 kB/s | 19 kB 00:00 2024-09-15T21:21:22.603 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 19 kB 00:00 2024-09-15T21:21:23.034 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-15T21:21:23.857 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-15T21:21:28.573 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-15T21:21:29.496 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-09-15T21:21:30.596 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T21:21:30.597 INFO:teuthology.orchestra.run.smithi192.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T21:21:30.697 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:21:30.708 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:21:30.708 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-15T21:21:30.708 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:21:30.708 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-09-15T21:21:30.709 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:21:30.710 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 79 M 2024-09-15T21:21:30.710 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 126 M 2024-09-15T21:21:30.710 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:21:30.858 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 241 kB/s | 23 kB 00:00 2024-09-15T21:21:31.175 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-09-15T21:21:31.475 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 46 MB/s | 29 MB 00:00 2024-09-15T21:21:31.594 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T21:21:31.594 INFO:teuthology.orchestra.run.smithi019.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T21:21:31.684 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:21:31.700 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:21:31.701 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:21:31.701 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2024-09-15T21:21:31.701 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:21:31.701 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 79 M 2024-09-15T21:21:31.701 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 126 M 2024-09-15T21:21:31.701 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:21:31.892 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 31 MB/s | 35 MB 00:01 2024-09-15T21:21:31.893 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:21:31.893 INFO:teuthology.orchestra.run.smithi192.stdout:Total 67 MB/s | 79 MB 00:01 2024-09-15T21:21:32.064 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 212 kB/s | 23 kB 00:00 2024-09-15T21:21:32.456 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-15T21:21:32.576 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:21:32.848 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 36 MB/s | 29 MB 00:00 2024-09-15T21:21:32.928 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:21:32.929 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:21:33.148 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-09-15T21:21:33.149 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:21:33.149 INFO:teuthology.orchestra.run.smithi019.stdout:Total 54 MB/s | 79 MB 00:01 2024-09-15T21:21:33.855 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:21:33.883 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:21:33.883 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:21:34.228 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:21:34.228 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:21:35.241 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:21:35.241 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:21:36.321 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:21:36.959 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:21:37.052 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:21:37.816 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:21:38.476 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:21:38.558 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:21:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:21:39.990 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:21:41.361 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:21:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:21:44.818 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:21:44.857 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:21:46.608 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:21:46.653 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:21:49.715 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:21:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:27.770 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:27.811 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:28.135 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:28.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:22:28.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:22:28.135 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:22:29.002 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:22:29.358 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:22:29.437 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:22:29.437 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:22:29.438 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T21:22:29.438 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:22:29.438 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:22:29.438 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-15T21:22:29.438 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:22:29.438 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:22:29.497 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:22:29.497 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:22:29.498 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2024-09-15T21:22:29.557 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2024-09-15T21:22:29.557 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2024-09-15T21:22:29.557 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2024-09-15T21:22:29.557 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 82.el9 2024-09-15T21:22:29.557 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2024-09-15T21:22:29.557 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8274670 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2024-09-15T21:22:29.558 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:22:29.559 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T21:22:29.559 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T21:22:29.559 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T21:22:29.559 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2024-09-15T21:22:29.559 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:22:29.559 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T21:22:29.560 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T21:22:29.560 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 5.14.0-505.el9.x86_64 2024-09-15T21:22:29.560 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T21:22:30.179 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2024-09-15T21:22:31.109 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:31.151 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:22:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:22:31.424 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:22:31.514 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T21:22:31.547 INFO:teuthology.orchestra.run.smithi192.stderr:done 2024-09-15T21:22:31.549 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-09-15T21:22:31.565 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.9wQlX3WLDo 2024-09-15T21:22:31.566 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9wQlX3WLDo 2024-09-15T21:22:31.633 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.9wQlX3WLDo 2024-09-15T21:22:31.770 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.9wQlX3WLDo is 6KB 2024-09-15T21:22:31.820 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.9wQlX3WLDo 2024-09-15T21:22:31.834 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T21:22:31.903 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T21:22:31.903 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T21:22:31.903 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T21:22:31.903 INFO:teuthology.orchestra.run.smithi192.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T21:22:31.904 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T21:22:32.069 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2024-09-15T21:22:32.295 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:22:32.295 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:22:32.295 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:22:32.295 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T21:22:32.296 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T21:22:32.296 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T21:22:32.296 INFO:teuthology.orchestra.run.smithi019.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T21:22:32.296 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:22:32.296 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:22:32.646 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:22:32.740 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:22:32.740 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:22:32.740 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T21:22:32.741 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:22:32.741 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:22:32.741 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T21:22:32.741 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi019.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:22:32.741 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:22:32.799 INFO:teuthology.orchestra.run.smithi019.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:22:32.800 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:22:32.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo rpm -qi grub2-tools 2024-09-15T21:22:32.834 INFO:teuthology.orchestra.run.smithi019.stdout:Name : grub2-tools 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Epoch : 1 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Version : 2.06 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Release : 82.el9 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Architecture: x86_64 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Group : Unspecified 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Size : 8274670 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:License : GPLv3+ 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T21:22:32.835 INFO:teuthology.orchestra.run.smithi019.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:Packager : builder@centos.org 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:Vendor : CentOS 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:Summary : Support tools for GRUB. 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:Description : 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:hardware devices. 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:22:32.836 INFO:teuthology.orchestra.run.smithi019.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T21:22:32.837 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T21:22:32.837 INFO:teuthology.task.kernel:Updating grub on smithi019 to boot 5.14.0-505.el9.x86_64 2024-09-15T21:22:32.838 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T21:22:33.590 INFO:teuthology.orchestra.run.smithi019.stderr:Generating grub configuration file ... 2024-09-15T21:22:34.913 INFO:teuthology.orchestra.run.smithi019.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T21:22:34.945 INFO:teuthology.orchestra.run.smithi019.stderr:done 2024-09-15T21:22:34.947 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-09-15T21:22:35.002 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.d46Reta8c3 2024-09-15T21:22:35.002 DEBUG:teuthology.orchestra.run.smithi019:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.d46Reta8c3 2024-09-15T21:22:35.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.d46Reta8c3 2024-09-15T21:22:35.216 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.d46Reta8c3 is 6KB 2024-09-15T21:22:35.265 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.d46Reta8c3 2024-09-15T21:22:35.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T21:22:35.343 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T21:22:35.343 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T21:22:35.343 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T21:22:35.343 INFO:teuthology.orchestra.run.smithi019.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T21:22:35.344 DEBUG:teuthology.orchestra.run.smithi019:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T21:22:35.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo shutdown -r now 2024-09-15T21:22:35.577 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2024-09-15T21:23:02.097 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-15T21:23:02.097 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-15T21:23:02.098 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:23:05.505 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T21:23:05.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-15T21:23:05.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:23:23.837 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-09-15T21:23:32.845 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-15T21:23:32.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:23:35.901 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.19 2024-09-15T21:23:36.637 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-09-15T21:23:45.643 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-15T21:23:45.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:23:47.901 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-15T21:23:47.902 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:24:04.033 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.192 2024-09-15T21:24:16.045 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-15T21:24:16.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:24:16.424 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-09-15T21:24:16.825 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-15T21:24:16.825 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-15T21:24:16.825 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2024-09-15T21:24:16.840 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-505.el9.x86_64 2024-09-15T21:24:16.840 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T21:24:16.840 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:24:16.841 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T21:24:17.842 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-15T21:24:17.842 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T21:24:17.929 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2024-09-15T21:24:17.982 DEBUG:teuthology.parallel:result is None 2024-09-15T21:24:19.834 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-09-15T21:24:20.188 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-15T21:24:20.189 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T21:24:20.189 DEBUG:teuthology.orchestra.run.smithi019:> uname -r 2024-09-15T21:24:20.205 INFO:teuthology.orchestra.run.smithi019.stdout:5.14.0-505.el9.x86_64 2024-09-15T21:24:20.205 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T21:24:20.205 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:24:20.205 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T21:24:21.206 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T21:24:21.207 DEBUG:teuthology.orchestra.run.smithi019:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T21:24:21.276 INFO:teuthology.orchestra.run.smithi019.stdout:ttyS1 2024-09-15T21:24:21.297 DEBUG:teuthology.parallel:result is None 2024-09-15T21:24:21.298 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T21:24:21.305 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T21:24:21.305 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:24:21.308 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:24:21.342 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T21:24:21.349 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T21:24:21.355 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T21:24:21.355 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:24:21.367 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:24:21.464 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T21:24:21.470 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T21:24:21.470 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:24:21.473 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:24:21.510 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:24:21.522 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:24:21.590 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:24:21.607 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:24:21.610 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T21:24:21.621 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T21:24:21.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:24:21.627 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:24:21.697 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T21:24:21.710 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T21:24:21.710 DEBUG:teuthology.orchestra.run.smithi019:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:24:21.718 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:24:21.768 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:24:21.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:24:21.869 DEBUG:teuthology.orchestra.run.smithi019:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:24:21.951 DEBUG:teuthology.orchestra.run.smithi019:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:24:21.995 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:24:21.995 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:24:22.062 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:24:22.144 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:24:22.213 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:24:22.268 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:24:22.311 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:24:22.311 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:24:22.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo service rsyslog restart 2024-09-15T21:24:22.381 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2024-09-15T21:24:22.425 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:24:22.465 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:24:22.905 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T21:24:22.913 INFO:teuthology.task.internal:Starting timer... 2024-09-15T21:24:22.913 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T21:24:22.923 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T21:24:22.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo service auditd rotate 2024-09-15T21:24:22.989 INFO:teuthology.orchestra.run.smithi019.stdout:Rotating logs: 2024-09-15T21:24:22.991 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2024-09-15T21:24:23.036 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2024-09-15T21:24:23.038 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T21:24:23.038 DEBUG:teuthology.orchestra.run.smithi019:> /usr/sbin/getenforce 2024-09-15T21:24:23.074 INFO:teuthology.orchestra.run.smithi019.stdout:Permissive 2024-09-15T21:24:23.075 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2024-09-15T21:24:23.156 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2024-09-15T21:24:23.157 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi019.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2024-09-15T21:24:23.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T21:24:23.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:24:23.185 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T21:24:23.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:24:23.213 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T21:24:23.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:24:23.257 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:24:23.295 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T21:24:23.304 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T21:24:23.307 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T21:24:23.433 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T21:24:23.444 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T21:24:23.445 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi019.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:28:14.214 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi019.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2024-09-15T21:28:14.215 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-15T21:28:14.216 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi019.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:28:14.293 DEBUG:teuthology.orchestra.run.smithi019:> true 2024-09-15T21:28:14.372 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi019.front.sepia.ceph.com' 2024-09-15T21:28:14.372 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-15T21:28:14.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:28:14.450 DEBUG:teuthology.orchestra.run.smithi192:> true 2024-09-15T21:28:14.533 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2024-09-15T21:28:14.533 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:28:14.544 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:28:14.544 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:28:14.545 DEBUG:teuthology.orchestra.run.smithi019:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:28:14.547 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:28:14.547 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:28:14.576 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:28:14.589 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:28:14.605 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:28:14.614 INFO:teuthology.orchestra.run.smithi019.stderr:sudo: ntpd: command not found 2024-09-15T21:28:14.619 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:28:14.626 INFO:teuthology.orchestra.run.smithi019.stdout:506 Cannot talk to daemon 2024-09-15T21:28:14.641 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:28:14.643 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2024-09-15T21:28:14.655 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2024-09-15T21:28:14.655 INFO:teuthology.orchestra.run.smithi019.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:28:14.669 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:28:14.683 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:28:14.718 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:28:14.722 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:28:14.723 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-09-15T21:28:14.723 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:28:14.723 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:28:14.723 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:28:14.723 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:28:14.748 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:28:14.752 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:28:14.752 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-15T21:28:14.752 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:28:14.752 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:28:14.752 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:28:14.752 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:28:14.753 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:28:14.760 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:28:14.760 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}} 2024-09-15T21:28:14.760 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'} 2024-09-15T21:28:14.760 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:28:14.765 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:28:14.765 INFO:teuthology.task.install:extra packages: [] 2024-09-15T21:28:14.766 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'tag': None, 'wait_for_package': False} 2024-09-15T21:28:14.766 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:28:14.768 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'tag': None, 'wait_for_package': False} 2024-09-15T21:28:14.768 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:28:14.909 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/ 2024-09-15T21:28:14.909 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4930.gfc3e0bff 2024-09-15T21:28:14.921 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/ 2024-09-15T21:28:14.921 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4930.gfc3e0bff 2024-09-15T21:28:15.023 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:28:15.024 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:28:15.024 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:28:15.038 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:28:15.039 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:28:15.039 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:28:15.051 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T21:28:15.051 DEBUG:teuthology.orchestra.run.smithi019:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/fc3e0bff7468fd55478adbae7f75508b749124b6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:28:15.064 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T21:28:15.064 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/fc3e0bff7468fd55478adbae7f75508b749124b6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:28:15.118 DEBUG:teuthology.orchestra.run.smithi019:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:28:15.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:28:15.195 DEBUG:teuthology.orchestra.run.smithi019:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:28:15.204 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:28:15.260 INFO:teuthology.orchestra.run.smithi019.stdout:check_obsoletes = 1 2024-09-15T21:28:15.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum clean all 2024-09-15T21:28:15.268 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2024-09-15T21:28:15.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2024-09-15T21:28:15.498 INFO:teuthology.orchestra.run.smithi192.stdout:45 files removed 2024-09-15T21:28:15.502 INFO:teuthology.orchestra.run.smithi019.stdout:45 files removed 2024-09-15T21:28:15.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw 2024-09-15T21:28:15.530 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-radosgw 2024-09-15T21:28:16.074 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 320 kB/s | 84 kB 00:00 2024-09-15T21:28:16.106 INFO:teuthology.orchestra.run.smithi019.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-15T21:28:16.383 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-15T21:28:16.398 INFO:teuthology.orchestra.run.smithi019.stdout:ceph noarch packages 98 kB/s | 24 kB 00:00 2024-09-15T21:28:16.659 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-15T21:28:16.682 INFO:teuthology.orchestra.run.smithi019.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-15T21:28:18.642 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.2 MB 00:01 2024-09-15T21:28:19.390 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.2 MB 00:02 2024-09-15T21:28:20.777 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-15T21:28:21.250 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-15T21:28:25.202 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-15T21:28:25.967 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.5 MB 00:00 2024-09-15T21:28:26.570 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 139 kB/s | 19 kB 00:00 2024-09-15T21:28:27.535 INFO:teuthology.orchestra.run.smithi019.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 19 kB 00:00 2024-09-15T21:28:27.932 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-15T21:28:28.805 INFO:teuthology.orchestra.run.smithi019.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-15T21:28:33.371 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-15T21:28:34.545 INFO:teuthology.orchestra.run.smithi019.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-09-15T21:28:35.832 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:28:35.844 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:28:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:28:35.844 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:28:35.844 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:28:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 11 M 2024-09-15T21:28:35.844 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2024-09-15T21:28:35.844 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.5 M 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.2 M 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 5.5 M 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 22 M 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 25 k 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 755 k 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-15T21:28:35.845 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 508 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 5.5 M 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 45 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 143 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 161 k 2024-09-15T21:28:35.846 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 320 k 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 305 k 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 99 k 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout:Install 31 Packages 2024-09-15T21:28:35.847 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 2 Packages 2024-09-15T21:28:35.848 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:28:35.848 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 62 M 2024-09-15T21:28:35.848 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:28:37.032 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:28:37.043 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:28:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:28:37.043 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:28:37.043 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:28:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 11 M 2024-09-15T21:28:37.043 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrading: 2024-09-15T21:28:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: librados2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.5 M 2024-09-15T21:28:37.043 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.2 M 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 5.5 M 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 22 M 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 25 k 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 755 k 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:28:37.044 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 508 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 5.5 M 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 45 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 143 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 161 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 320 k 2024-09-15T21:28:37.045 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 305 k 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 99 k 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout:Installing weak dependencies: 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout:Install 31 Packages 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout:Upgrade 2 Packages 2024-09-15T21:28:37.046 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:28:37.047 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 62 M 2024-09-15T21:28:37.047 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:28:37.923 INFO:teuthology.orchestra.run.smithi192.stdout:(1/33): ceph-base-19.3.0-4930.gfc3e0bff.el9.x86 2.9 MB/s | 5.5 MB 00:01 2024-09-15T21:28:37.998 INFO:teuthology.orchestra.run.smithi192.stdout:(2/33): ceph-selinux-19.3.0-4930.gfc3e0bff.el9. 336 kB/s | 25 kB 00:00 2024-09-15T21:28:38.190 INFO:teuthology.orchestra.run.smithi192.stdout:(3/33): libcephfs2-19.3.0-4930.gfc3e0bff.el9.x8 3.8 MB/s | 755 kB 00:00 2024-09-15T21:28:38.298 INFO:teuthology.orchestra.run.smithi192.stdout:(4/33): libradosstriper1-19.3.0-4930.gfc3e0bff. 4.6 MB/s | 508 kB 00:00 2024-09-15T21:28:39.015 INFO:teuthology.orchestra.run.smithi192.stdout:(5/33): librgw2-19.3.0-4930.gfc3e0bff.el9.x86_6 7.7 MB/s | 5.5 MB 00:00 2024-09-15T21:28:39.065 INFO:teuthology.orchestra.run.smithi192.stdout:(6/33): python3-ceph-argparse-19.3.0-4930.gfc3e 903 kB/s | 45 kB 00:00 2024-09-15T21:28:39.124 INFO:teuthology.orchestra.run.smithi192.stdout:(7/33): python3-ceph-common-19.3.0-4930.gfc3e0b 2.4 MB/s | 143 kB 00:00 2024-09-15T21:28:39.191 INFO:teuthology.orchestra.run.smithi192.stdout:(8/33): python3-cephfs-19.3.0-4930.gfc3e0bff.el 2.4 MB/s | 161 kB 00:00 2024-09-15T21:28:39.291 INFO:teuthology.orchestra.run.smithi192.stdout:(9/33): python3-rados-19.3.0-4930.gfc3e0bff.el9 3.1 MB/s | 320 kB 00:00 2024-09-15T21:28:39.391 INFO:teuthology.orchestra.run.smithi192.stdout:(10/33): python3-rbd-19.3.0-4930.gfc3e0bff.el9. 3.0 MB/s | 305 kB 00:00 2024-09-15T21:28:39.458 INFO:teuthology.orchestra.run.smithi192.stdout:(11/33): python3-rgw-19.3.0-4930.gfc3e0bff.el9. 1.5 MB/s | 99 kB 00:00 2024-09-15T21:28:39.559 INFO:teuthology.orchestra.run.smithi192.stdout:(12/33): boost-program-options-1.75.0-8.el9.x86 1.0 MB/s | 107 kB 00:00 2024-09-15T21:28:39.617 INFO:teuthology.orchestra.run.smithi192.stdout:(13/33): libnbd-1.20.2-2.el9.x86_64.rpm 2.9 MB/s | 171 kB 00:00 2024-09-15T21:28:39.659 INFO:teuthology.orchestra.run.smithi192.stdout:(14/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-09-15T21:28:39.693 INFO:teuthology.orchestra.run.smithi192.stdout:(15/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-09-15T21:28:39.751 INFO:teuthology.orchestra.run.smithi192.stdout:(16/33): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-15T21:28:39.801 INFO:teuthology.orchestra.run.smithi192.stdout:(17/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-15T21:28:39.804 INFO:teuthology.orchestra.run.smithi019.stdout:(1/33): ceph-base-19.3.0-4930.gfc3e0bff.el9.x86 2.2 MB/s | 5.5 MB 00:02 2024-09-15T21:28:39.835 INFO:teuthology.orchestra.run.smithi192.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 42 kB 00:00 2024-09-15T21:28:39.863 INFO:teuthology.orchestra.run.smithi019.stdout:(2/33): ceph-selinux-19.3.0-4930.gfc3e0bff.el9. 433 kB/s | 25 kB 00:00 2024-09-15T21:28:39.885 INFO:teuthology.orchestra.run.smithi192.stdout:(19/33): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-15T21:28:39.919 INFO:teuthology.orchestra.run.smithi192.stdout:(20/33): qatlib-service-24.02.0-1.el9.x86_64.rp 1.1 MB/s | 36 kB 00:00 2024-09-15T21:28:39.969 INFO:teuthology.orchestra.run.smithi192.stdout:(21/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-15T21:28:40.019 INFO:teuthology.orchestra.run.smithi192.stdout:(22/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 6.0 MB/s | 308 kB 00:00 2024-09-15T21:28:40.046 INFO:teuthology.orchestra.run.smithi019.stdout:(3/33): libcephfs2-19.3.0-4930.gfc3e0bff.el9.x8 4.0 MB/s | 755 kB 00:00 2024-09-15T21:28:40.172 INFO:teuthology.orchestra.run.smithi019.stdout:(4/33): libradosstriper1-19.3.0-4930.gfc3e0bff. 4.0 MB/s | 508 kB 00:00 2024-09-15T21:28:40.188 INFO:teuthology.orchestra.run.smithi192.stdout:(23/33): libarrow-9.0.0-11.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2024-09-15T21:28:40.203 INFO:teuthology.orchestra.run.smithi192.stdout:(24/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.5 MB/s | 26 kB 00:00 2024-09-15T21:28:40.220 INFO:teuthology.orchestra.run.smithi192.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-15T21:28:40.246 INFO:teuthology.orchestra.run.smithi192.stdout:(26/33): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-15T21:28:40.287 INFO:teuthology.orchestra.run.smithi192.stdout:(27/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 20 MB/s | 839 kB 00:00 2024-09-15T21:28:40.321 INFO:teuthology.orchestra.run.smithi192.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-15T21:28:40.413 INFO:teuthology.orchestra.run.smithi192.stdout:(29/33): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-09-15T21:28:40.542 INFO:teuthology.orchestra.run.smithi019.stdout:(5/33): ceph-radosgw-19.3.0-4930.gfc3e0bff.el9. 3.4 MB/s | 11 MB 00:03 2024-09-15T21:28:40.600 INFO:teuthology.orchestra.run.smithi019.stdout:(6/33): python3-ceph-argparse-19.3.0-4930.gfc3e 826 kB/s | 45 kB 00:00 2024-09-15T21:28:40.656 INFO:teuthology.orchestra.run.smithi019.stdout:(7/33): python3-ceph-common-19.3.0-4930.gfc3e0b 2.4 MB/s | 143 kB 00:00 2024-09-15T21:28:40.672 INFO:teuthology.orchestra.run.smithi192.stdout:(30/33): ceph-common-19.3.0-4930.gfc3e0bff.el9. 4.6 MB/s | 22 MB 00:04 2024-09-15T21:28:40.756 INFO:teuthology.orchestra.run.smithi019.stdout:(8/33): librgw2-19.3.0-4930.gfc3e0bff.el9.x86_6 9.4 MB/s | 5.5 MB 00:00 2024-09-15T21:28:40.781 INFO:teuthology.orchestra.run.smithi019.stdout:(9/33): python3-cephfs-19.3.0-4930.gfc3e0bff.el 1.3 MB/s | 161 kB 00:00 2024-09-15T21:28:40.815 INFO:teuthology.orchestra.run.smithi019.stdout:(10/33): python3-rados-19.3.0-4930.gfc3e0bff.el 5.4 MB/s | 320 kB 00:00 2024-09-15T21:28:40.849 INFO:teuthology.orchestra.run.smithi019.stdout:(11/33): python3-rbd-19.3.0-4930.gfc3e0bff.el9. 4.4 MB/s | 305 kB 00:00 2024-09-15T21:28:40.874 INFO:teuthology.orchestra.run.smithi019.stdout:(12/33): python3-rgw-19.3.0-4930.gfc3e0bff.el9. 1.7 MB/s | 99 kB 00:00 2024-09-15T21:28:41.064 INFO:teuthology.orchestra.run.smithi192.stdout:(31/33): ceph-radosgw-19.3.0-4930.gfc3e0bff.el9 2.2 MB/s | 11 MB 00:05 2024-09-15T21:28:41.108 INFO:teuthology.orchestra.run.smithi019.stdout:(13/33): ceph-common-19.3.0-4930.gfc3e0bff.el9. 5.8 MB/s | 22 MB 00:03 2024-09-15T21:28:41.114 INFO:teuthology.orchestra.run.smithi192.stdout:(32/33): librados2-19.3.0-4930.gfc3e0bff.el9.x8 4.9 MB/s | 3.5 MB 00:00 2024-09-15T21:28:41.164 INFO:teuthology.orchestra.run.smithi192.stdout:(33/33): librbd1-19.3.0-4930.gfc3e0bff.el9.x86_ 6.6 MB/s | 3.2 MB 00:00 2024-09-15T21:28:41.168 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:28:41.168 INFO:teuthology.orchestra.run.smithi192.stdout:Total 12 MB/s | 62 MB 00:05 2024-09-15T21:28:41.191 INFO:teuthology.orchestra.run.smithi019.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 312 kB/s | 107 kB 00:00 2024-09-15T21:28:41.250 INFO:teuthology.orchestra.run.smithi019.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 454 kB/s | 171 kB 00:00 2024-09-15T21:28:41.266 INFO:teuthology.orchestra.run.smithi019.stdout:(16/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 600 kB/s | 45 kB 00:00 2024-09-15T21:28:41.283 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:28:41.300 INFO:teuthology.orchestra.run.smithi019.stdout:(17/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 834 kB/s | 160 kB 00:00 2024-09-15T21:28:41.320 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:28:41.320 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:28:41.350 INFO:teuthology.orchestra.run.smithi019.stdout:(18/33): python3-prettytable-0.7.2-27.el9.noarc 846 kB/s | 42 kB 00:00 2024-09-15T21:28:41.376 INFO:teuthology.orchestra.run.smithi019.stdout:(19/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-09-15T21:28:41.409 INFO:teuthology.orchestra.run.smithi019.stdout:(20/33): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-09-15T21:28:41.435 INFO:teuthology.orchestra.run.smithi019.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 611 kB/s | 36 kB 00:00 2024-09-15T21:28:41.460 INFO:teuthology.orchestra.run.smithi019.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-09-15T21:28:41.494 INFO:teuthology.orchestra.run.smithi019.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-15T21:28:41.544 INFO:teuthology.orchestra.run.smithi019.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-15T21:28:41.569 INFO:teuthology.orchestra.run.smithi019.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-15T21:28:41.594 INFO:teuthology.orchestra.run.smithi019.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 285 kB/s | 26 kB 00:00 2024-09-15T21:28:41.628 INFO:teuthology.orchestra.run.smithi019.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-15T21:28:41.670 INFO:teuthology.orchestra.run.smithi019.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-15T21:28:41.729 INFO:teuthology.orchestra.run.smithi019.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 839 kB 00:00 2024-09-15T21:28:41.820 INFO:teuthology.orchestra.run.smithi019.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-15T21:28:41.896 INFO:teuthology.orchestra.run.smithi019.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-15T21:28:42.018 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:28:42.018 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:28:42.055 INFO:teuthology.orchestra.run.smithi019.stdout:(32/33): librados2-19.3.0-4930.gfc3e0bff.el9.x8 11 MB/s | 3.5 MB 00:00 2024-09-15T21:28:42.113 INFO:teuthology.orchestra.run.smithi019.stdout:(33/33): librbd1-19.3.0-4930.gfc3e0bff.el9.x86_ 11 MB/s | 3.2 MB 00:00 2024-09-15T21:28:42.117 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:28:42.118 INFO:teuthology.orchestra.run.smithi019.stdout:Total 12 MB/s | 62 MB 00:05 2024-09-15T21:28:42.229 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:28:42.269 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:28:42.269 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:28:42.981 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:28:42.981 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:28:43.526 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:28:43.606 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-15T21:28:43.636 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-15T21:28:43.669 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-15T21:28:43.692 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-15T21:28:43.717 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-15T21:28:43.911 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-15T21:28:43.965 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-15T21:28:44.369 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-15T21:28:44.496 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:28:44.562 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:28:44.594 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:28:44.644 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-15T21:28:44.677 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-15T21:28:44.687 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 8/35 2024-09-15T21:28:44.702 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-15T21:28:44.723 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 9/35 2024-09-15T21:28:44.733 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-15T21:28:44.761 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-15T21:28:44.785 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 9/35 2024-09-15T21:28:44.822 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-15T21:28:44.874 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9. 11/35 2024-09-15T21:28:44.911 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-15T21:28:44.945 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 12/35 2024-09-15T21:28:44.964 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-15T21:28:44.974 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 13/35 2024-09-15T21:28:45.039 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 13/35 2024-09-15T21:28:45.073 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-15T21:28:45.184 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-15T21:28:45.215 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-15T21:28:45.262 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-15T21:28:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-15T21:28:45.326 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-15T21:28:45.360 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-15T21:28:45.400 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-15T21:28:45.405 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-15T21:28:45.436 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:28:45.530 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:28:45.620 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 8/35 2024-09-15T21:28:45.646 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 9/35 2024-09-15T21:28:45.692 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 9/35 2024-09-15T21:28:45.721 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-15T21:28:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-15T21:28:45.773 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 23/35 2024-09-15T21:28:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9. 11/35 2024-09-15T21:28:45.844 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 12/35 2024-09-15T21:28:45.864 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 23/35 2024-09-15T21:28:45.873 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 13/35 2024-09-15T21:28:45.903 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 24/35 2024-09-15T21:28:45.933 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 13/35 2024-09-15T21:28:45.939 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-15T21:28:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-15T21:28:46.107 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-15T21:28:46.140 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-15T21:28:46.190 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-15T21:28:46.221 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-15T21:28:46.251 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-15T21:28:46.284 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-15T21:28:46.321 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-15T21:28:46.437 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-15T21:28:46.467 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 27/35 2024-09-15T21:28:46.537 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 27/35 2024-09-15T21:28:46.607 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 28/35 2024-09-15T21:28:46.636 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x8 29/35 2024-09-15T21:28:46.655 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-15T21:28:46.697 INFO:teuthology.orchestra.run.smithi019.stdout: Upgrading : librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 23/35 2024-09-15T21:28:46.798 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 23/35 2024-09-15T21:28:46.827 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 24/35 2024-09-15T21:28:46.871 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-15T21:28:47.389 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-15T21:28:47.416 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 27/35 2024-09-15T21:28:47.471 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 27/35 2024-09-15T21:28:47.543 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 28/35 2024-09-15T21:28:47.593 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x8 29/35 2024-09-15T21:28:48.970 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:28:49.007 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:28:49.598 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:28:49.635 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 31/35 2024-09-15T21:28:49.778 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 31/35 2024-09-15T21:28:49.779 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:28:49.779 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:28:49.779 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:28:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:28:50.058 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:28:50.098 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:28:50.694 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:28:50.735 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 31/35 2024-09-15T21:28:50.826 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 31/35 2024-09-15T21:28:50.826 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:28:50.827 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:28:50.827 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:28:50.861 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:28:59.833 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:28:59.833 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2024-09-15T21:28:59.833 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2024-09-15T21:28:59.833 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2024-09-15T21:28:59.833 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2024-09-15T21:28:59.833 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2024-09-15T21:28:59.833 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2024-09-15T21:28:59.834 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2024-09-15T21:28:59.834 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:00.870 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/35 2024-09-15T21:29:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/35 2024-09-15T21:29:00.896 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:00.896 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:29:00.896 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:29:00.896 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:29:00.896 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:00.898 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T21:29:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T21:29:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:29:01.256 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:29:01.257 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /sys 2024-09-15T21:29:01.257 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /proc 2024-09-15T21:29:01.257 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /mnt 2024-09-15T21:29:01.257 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /var/tmp 2024-09-15T21:29:01.257 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /home 2024-09-15T21:29:01.257 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /root 2024-09-15T21:29:01.257 INFO:teuthology.orchestra.run.smithi019.stdout:skipping the directory /tmp 2024-09-15T21:29:01.257 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:02.232 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:29:02.232 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 5/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 6/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9. 8/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x8 9/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 10/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 11/35 2024-09-15T21:29:02.233 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 12/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 13/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-15T21:29:02.234 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-15T21:29:02.235 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-15T21:29:02.235 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-15T21:29:02.235 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-15T21:29:02.236 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-15T21:29:02.236 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-15T21:29:02.236 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-15T21:29:02.236 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-15T21:29:02.236 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:29:02.236 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-15T21:29:02.236 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/35 2024-09-15T21:29:02.287 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/35 2024-09-15T21:29:02.313 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/35 2024-09-15T21:29:02.314 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:02.314 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:29:02.314 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:29:02.314 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:29:02.314 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:02.316 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T21:29:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T21:29:02.406 INFO:teuthology.orchestra.run.smithi019.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.287 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:29:03.288 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T21:29:03.289 INFO:teuthology.orchestra.run.smithi192.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:29:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:29:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:29:03.290 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:03.290 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:03.633 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-test 2024-09-15T21:29:03.712 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:29:03.712 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 5/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 6/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9. 8/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x8 9/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 10/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 11/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 12/35 2024-09-15T21:29:03.713 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 13/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-15T21:29:03.714 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-15T21:29:03.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-15T21:29:03.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-15T21:29:03.715 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-15T21:29:03.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-15T21:29:03.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-15T21:29:03.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-15T21:29:03.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-15T21:29:03.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:29:03.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-15T21:29:03.716 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/35 2024-09-15T21:29:04.203 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:31 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:04.344 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:04.344 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:04.344 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:29:04.344 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:04.345 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:29:04.345 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 46 M 2024-09-15T21:29:04.348 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:29:04.348 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 161 k 2024-09-15T21:29:04.348 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:29:04.348 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:29:04.349 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:04.349 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:29:04.349 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:04.349 INFO:teuthology.orchestra.run.smithi192.stdout:Install 4 Packages 2024-09-15T21:29:04.349 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:04.349 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 46 M 2024-09-15T21:29:04.349 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 201 M 2024-09-15T21:29:04.350 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:29:04.512 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:29:04.512 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:04.512 INFO:teuthology.orchestra.run.smithi019.stdout:Upgraded: 2024-09-15T21:29:04.512 INFO:teuthology.orchestra.run.smithi019.stdout: librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-15T21:29:04.513 INFO:teuthology.orchestra.run.smithi019.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:29:04.514 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:04.515 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:04.594 INFO:teuthology.orchestra.run.smithi192.stdout:(1/4): libcephsqlite-19.3.0-4930.gfc3e0bff.el9. 722 kB/s | 161 kB 00:00 2024-09-15T21:29:04.870 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-test 2024-09-15T21:29:04.969 INFO:teuthology.orchestra.run.smithi192.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-09-15T21:29:05.003 INFO:teuthology.orchestra.run.smithi192.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 481 kB/s | 304 kB 00:00 2024-09-15T21:29:05.468 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:31 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:05.607 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:05.608 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:29:05.608 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 46 M 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 161 k 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Install 4 Packages 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 46 M 2024-09-15T21:29:05.609 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 201 M 2024-09-15T21:29:05.610 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:29:05.895 INFO:teuthology.orchestra.run.smithi019.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-15T21:29:05.945 INFO:teuthology.orchestra.run.smithi019.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-15T21:29:05.978 INFO:teuthology.orchestra.run.smithi019.stdout:(3/4): libcephsqlite-19.3.0-4930.gfc3e0bff.el9. 603 kB/s | 161 kB 00:00 2024-09-15T21:29:06.011 INFO:teuthology.orchestra.run.smithi192.stdout:(4/4): ceph-test-19.3.0-4930.gfc3e0bff.el9.x86_ 28 MB/s | 46 MB 00:01 2024-09-15T21:29:06.013 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:06.014 INFO:teuthology.orchestra.run.smithi192.stdout:Total 28 MB/s | 46 MB 00:01 2024-09-15T21:29:06.041 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:29:06.061 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:29:06.061 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:29:06.429 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:29:06.429 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:29:06.945 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:29:07.018 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T21:29:07.061 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T21:29:07.130 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/4 2024-09-15T21:29:08.162 INFO:teuthology.orchestra.run.smithi019.stdout:(4/4): ceph-test-19.3.0-4930.gfc3e0bff.el9.x86_ 19 MB/s | 46 MB 00:02 2024-09-15T21:29:08.164 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:08.165 INFO:teuthology.orchestra.run.smithi019.stdout:Total 18 MB/s | 46 MB 00:02 2024-09-15T21:29:08.193 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:29:08.216 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:29:08.216 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:29:08.618 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:29:08.618 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:29:09.190 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:29:09.263 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T21:29:09.304 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T21:29:09.331 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/4 2024-09-15T21:29:11.476 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/4 2024-09-15T21:29:11.562 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/4 2024-09-15T21:29:13.868 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/4 2024-09-15T21:29:13.868 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/4 2024-09-15T21:29:13.868 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/4 2024-09-15T21:29:13.868 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T21:29:13.935 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/4 2024-09-15T21:29:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/4 2024-09-15T21:29:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T21:29:14.174 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:14.174 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:29:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:29:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:29:14.175 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:14.175 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:14.465 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph 2024-09-15T21:29:15.035 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:42 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:15.179 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:15.181 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 6.6 k 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 2.4 M 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 1.5 M 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 291 k 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 4.8 M 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 17 M 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:29:15.182 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:29:15.183 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:29:15.184 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:29:15.185 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:29:15.185 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:29:15.185 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:29:15.185 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:15.185 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:29:15.185 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:15.185 INFO:teuthology.orchestra.run.smithi192.stdout:Install 37 Packages 2024-09-15T21:29:15.185 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:15.186 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 31 M 2024-09-15T21:29:15.186 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 109 M 2024-09-15T21:29:15.186 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:29:15.921 INFO:teuthology.orchestra.run.smithi192.stdout:(1/37): ceph-19.3.0-4930.gfc3e0bff.el9.x86_64.r 47 kB/s | 6.6 kB 00:00 2024-09-15T21:29:16.096 INFO:teuthology.orchestra.run.smithi192.stdout:(2/37): ceph-mgr-19.3.0-4930.gfc3e0bff.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-15T21:29:16.171 INFO:teuthology.orchestra.run.smithi192.stdout:(3/37): ceph-mds-19.3.0-4930.gfc3e0bff.el9.x86_ 6.1 MB/s | 2.4 MB 00:00 2024-09-15T21:29:16.221 INFO:teuthology.orchestra.run.smithi192.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4930.gfc3e 5.7 MB/s | 291 kB 00:00 2024-09-15T21:29:16.397 INFO:teuthology.orchestra.run.smithi192.stdout:(5/37): ceph-mon-19.3.0-4930.gfc3e0bff.el9.x86_ 10 MB/s | 4.8 MB 00:00 2024-09-15T21:29:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/4 2024-09-15T21:29:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/4 2024-09-15T21:29:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/4 2024-09-15T21:29:16.410 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T21:29:16.455 INFO:teuthology.orchestra.run.smithi192.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-15T21:29:16.522 INFO:teuthology.orchestra.run.smithi192.stdout:(7/37): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-09-15T21:29:16.573 INFO:teuthology.orchestra.run.smithi192.stdout:(8/37): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-09-15T21:29:16.623 INFO:teuthology.orchestra.run.smithi192.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-09-15T21:29:16.681 INFO:teuthology.orchestra.run.smithi192.stdout:(10/37): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-15T21:29:16.723 INFO:teuthology.orchestra.run.smithi192.stdout:(11/37): python3-cryptography-36.0.1-4.el9.x86_ 3.8 MB/s | 1.2 MB 00:00 2024-09-15T21:29:16.749 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T21:29:16.749 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:16.749 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:29:16.749 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:16.749 INFO:teuthology.orchestra.run.smithi019.stdout: libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:16.749 INFO:teuthology.orchestra.run.smithi019.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:29:16.749 INFO:teuthology.orchestra.run.smithi019.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:29:16.750 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:16.750 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:16.832 INFO:teuthology.orchestra.run.smithi192.stdout:(12/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.6 MB/s | 172 kB 00:00 2024-09-15T21:29:16.857 INFO:teuthology.orchestra.run.smithi192.stdout:(13/37): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-09-15T21:29:17.016 INFO:teuthology.orchestra.run.smithi192.stdout:(14/37): ceph-osd-19.3.0-4930.gfc3e0bff.el9.x86 19 MB/s | 17 MB 00:00 2024-09-15T21:29:17.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph 2024-09-15T21:29:17.041 INFO:teuthology.orchestra.run.smithi192.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 166 kB/s | 35 kB 00:00 2024-09-15T21:29:17.058 INFO:teuthology.orchestra.run.smithi192.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 385 kB/s | 77 kB 00:00 2024-09-15T21:29:17.100 INFO:teuthology.orchestra.run.smithi192.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 710 kB/s | 42 kB 00:00 2024-09-15T21:29:17.142 INFO:teuthology.orchestra.run.smithi192.stdout:(18/37): python3-pytz-2021.1-5.el9.noarch.rpm 406 kB/s | 51 kB 00:00 2024-09-15T21:29:17.167 INFO:teuthology.orchestra.run.smithi192.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 204 kB/s | 22 kB 00:00 2024-09-15T21:29:17.209 INFO:teuthology.orchestra.run.smithi192.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-09-15T21:29:17.234 INFO:teuthology.orchestra.run.smithi192.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 472 kB/s | 43 kB 00:00 2024-09-15T21:29:17.260 INFO:teuthology.orchestra.run.smithi192.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 429 kB/s | 11 kB 00:00 2024-09-15T21:29:17.285 INFO:teuthology.orchestra.run.smithi192.stdout:(23/37): python3-cherrypy-18.6.1-2.el9.noarch.r 4.8 MB/s | 358 kB 00:00 2024-09-15T21:29:17.310 INFO:teuthology.orchestra.run.smithi192.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-15T21:29:17.335 INFO:teuthology.orchestra.run.smithi192.stdout:(25/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-15T21:29:17.352 INFO:teuthology.orchestra.run.smithi192.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 345 kB/s | 23 kB 00:00 2024-09-15T21:29:17.369 INFO:teuthology.orchestra.run.smithi192.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 330 kB/s | 19 kB 00:00 2024-09-15T21:29:17.386 INFO:teuthology.orchestra.run.smithi192.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-15T21:29:17.403 INFO:teuthology.orchestra.run.smithi192.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 1.4 MB/s | 46 kB 00:00 2024-09-15T21:29:17.428 INFO:teuthology.orchestra.run.smithi192.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-15T21:29:17.454 INFO:teuthology.orchestra.run.smithi192.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-09-15T21:29:17.479 INFO:teuthology.orchestra.run.smithi192.stdout:(32/37): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-09-15T21:29:17.496 INFO:teuthology.orchestra.run.smithi192.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-09-15T21:29:17.524 INFO:teuthology.orchestra.run.smithi192.stdout:(34/37): python3-tempora-5.0.0-2.el9.noarch.rpm 509 kB/s | 36 kB 00:00 2024-09-15T21:29:17.550 INFO:teuthology.orchestra.run.smithi192.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 3.2 MB/s | 230 kB 00:00 2024-09-15T21:29:17.575 INFO:teuthology.orchestra.run.smithi192.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-15T21:29:17.617 INFO:teuthology.orchestra.run.smithi192.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.5 MB/s | 427 kB 00:00 2024-09-15T21:29:17.621 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:17.621 INFO:teuthology.orchestra.run.smithi192.stdout:Total 13 MB/s | 31 MB 00:02 2024-09-15T21:29:17.651 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:43 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:17.771 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:29:17.797 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:17.799 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: ceph x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 6.6 k 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 2.4 M 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 1.5 M 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 291 k 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 4.8 M 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 17 M 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:29:17.800 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:29:17.801 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:29:17.802 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout:Install 37 Packages 2024-09-15T21:29:17.803 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:17.804 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 31 M 2024-09-15T21:29:17.804 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 109 M 2024-09-15T21:29:17.804 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:29:17.811 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:29:17.811 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:29:18.213 INFO:teuthology.orchestra.run.smithi019.stdout:(1/37): ceph-19.3.0-4930.gfc3e0bff.el9.x86_64.r 52 kB/s | 6.6 kB 00:00 2024-09-15T21:29:18.280 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:29:18.281 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:29:18.406 INFO:teuthology.orchestra.run.smithi019.stdout:(2/37): ceph-mgr-19.3.0-4930.gfc3e0bff.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-15T21:29:18.456 INFO:teuthology.orchestra.run.smithi019.stdout:(3/37): ceph-mds-19.3.0-4930.gfc3e0bff.el9.x86_ 6.4 MB/s | 2.4 MB 00:00 2024-09-15T21:29:18.515 INFO:teuthology.orchestra.run.smithi019.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4930.gfc3e 4.9 MB/s | 291 kB 00:00 2024-09-15T21:29:18.715 INFO:teuthology.orchestra.run.smithi019.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-15T21:29:18.868 INFO:teuthology.orchestra.run.smithi019.stdout:(6/37): python3-cryptography-36.0.1-4.el9.x86_6 8.0 MB/s | 1.2 MB 00:00 2024-09-15T21:29:18.935 INFO:teuthology.orchestra.run.smithi019.stdout:(7/37): ceph-mon-19.3.0-4930.gfc3e0bff.el9.x86_ 6.6 MB/s | 4.8 MB 00:00 2024-09-15T21:29:18.961 INFO:teuthology.orchestra.run.smithi019.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-15T21:29:19.144 INFO:teuthology.orchestra.run.smithi019.stdout:(9/37): ceph-osd-19.3.0-4930.gfc3e0bff.el9.x86_ 23 MB/s | 17 MB 00:00 2024-09-15T21:29:19.170 INFO:teuthology.orchestra.run.smithi019.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 598 kB/s | 125 kB 00:00 2024-09-15T21:29:19.228 INFO:teuthology.orchestra.run.smithi019.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 462 kB/s | 135 kB 00:00 2024-09-15T21:29:19.328 INFO:teuthology.orchestra.run.smithi019.stdout:(12/37): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2024-09-15T21:29:19.353 INFO:teuthology.orchestra.run.smithi019.stdout:(13/37): python3-urllib3-1.26.5-6.el9.noarch.rp 1.0 MB/s | 215 kB 00:00 2024-09-15T21:29:19.379 INFO:teuthology.orchestra.run.smithi019.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-15T21:29:19.404 INFO:teuthology.orchestra.run.smithi019.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 457 kB/s | 35 kB 00:00 2024-09-15T21:29:19.455 INFO:teuthology.orchestra.run.smithi019.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-15T21:29:19.480 INFO:teuthology.orchestra.run.smithi019.stdout:(17/37): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-09-15T21:29:19.525 INFO:teuthology.orchestra.run.smithi019.stdout:(18/37): python3-packaging-20.9-5.el9.noarch.rp 450 kB/s | 77 kB 00:00 2024-09-15T21:29:19.547 INFO:teuthology.orchestra.run.smithi019.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 242 kB/s | 22 kB 00:00 2024-09-15T21:29:19.606 INFO:teuthology.orchestra.run.smithi019.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-09-15T21:29:19.631 INFO:teuthology.orchestra.run.smithi019.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 412 kB/s | 43 kB 00:00 2024-09-15T21:29:19.664 INFO:teuthology.orchestra.run.smithi019.stdout:(22/37): python3-cherrypy-18.6.1-2.el9.noarch.r 6.2 MB/s | 358 kB 00:00 2024-09-15T21:29:19.667 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:29:19.690 INFO:teuthology.orchestra.run.smithi019.stdout:(23/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-09-15T21:29:19.715 INFO:teuthology.orchestra.run.smithi019.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-15T21:29:19.728 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T21:29:19.740 INFO:teuthology.orchestra.run.smithi019.stdout:(25/37): python3-jaraco-classes-3.2.1-5.el9.noa 353 kB/s | 18 kB 00:00 2024-09-15T21:29:19.760 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T21:29:19.766 INFO:teuthology.orchestra.run.smithi019.stdout:(26/37): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-09-15T21:29:19.791 INFO:teuthology.orchestra.run.smithi019.stdout:(27/37): python3-jaraco-functools-3.5.0-2.el9.n 257 kB/s | 19 kB 00:00 2024-09-15T21:29:19.797 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T21:29:19.816 INFO:teuthology.orchestra.run.smithi019.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 260 kB/s | 20 kB 00:00 2024-09-15T21:29:19.841 INFO:teuthology.orchestra.run.smithi019.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-09-15T21:29:19.868 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T21:29:19.875 INFO:teuthology.orchestra.run.smithi019.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 943 kB/s | 79 kB 00:00 2024-09-15T21:29:19.900 INFO:teuthology.orchestra.run.smithi019.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 281 kB/s | 16 kB 00:00 2024-09-15T21:29:19.926 INFO:teuthology.orchestra.run.smithi019.stdout:(32/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-15T21:29:19.929 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T21:29:19.951 INFO:teuthology.orchestra.run.smithi019.stdout:(33/37): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-09-15T21:29:19.962 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T21:29:19.977 INFO:teuthology.orchestra.run.smithi019.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-15T21:29:20.010 INFO:teuthology.orchestra.run.smithi019.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-15T21:29:20.035 INFO:teuthology.orchestra.run.smithi019.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-09-15T21:29:20.052 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T21:29:20.077 INFO:teuthology.orchestra.run.smithi019.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.3 MB/s | 427 kB 00:00 2024-09-15T21:29:20.080 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:20.080 INFO:teuthology.orchestra.run.smithi019.stdout:Total 14 MB/s | 31 MB 00:02 2024-09-15T21:29:20.124 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T21:29:20.165 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T21:29:20.200 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T21:29:20.234 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:29:20.238 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T21:29:20.272 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T21:29:20.278 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:29:20.279 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:29:20.306 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T21:29:20.346 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T21:29:20.389 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T21:29:20.423 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T21:29:20.472 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T21:29:20.504 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T21:29:20.577 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T21:29:20.724 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T21:29:20.802 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T21:29:20.820 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:29:20.821 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:29:20.856 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T21:29:20.895 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T21:29:20.944 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T21:29:21.001 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T21:29:21.186 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T21:29:21.235 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T21:29:21.311 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T21:29:21.400 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T21:29:21.444 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T21:29:21.568 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T21:29:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9. 32/37 2024-09-15T21:29:21.761 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/37 2024-09-15T21:29:21.788 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/37 2024-09-15T21:29:21.788 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:21.788 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:29:21.788 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:29:21.789 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:29:21.789 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:22.186 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:29:22.247 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T21:29:22.280 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T21:29:22.324 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T21:29:22.401 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T21:29:22.449 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T21:29:22.481 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T21:29:22.594 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T21:29:22.671 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T21:29:22.726 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T21:29:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T21:29:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T21:29:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T21:29:22.866 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T21:29:22.916 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T21:29:22.959 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T21:29:22.987 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T21:29:23.073 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T21:29:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T21:29:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T21:29:23.254 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/37 2024-09-15T21:29:23.282 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/37 2024-09-15T21:29:23.282 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:23.282 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:29:23.282 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:29:23.282 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:29:23.282 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:23.338 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T21:29:23.421 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T21:29:23.480 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T21:29:23.518 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T21:29:23.585 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T21:29:23.649 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T21:29:23.738 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 35/37 2024-09-15T21:29:23.762 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 35/37 2024-09-15T21:29:23.762 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:23.762 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:29:23.762 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:29:23.762 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:29:23.762 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:23.845 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T21:29:23.897 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T21:29:23.986 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T21:29:23.989 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 36/37 2024-09-15T21:29:24.012 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 36/37 2024-09-15T21:29:24.012 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:24.012 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:29:24.012 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:29:24.013 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:29:24.013 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:24.076 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 37/37 2024-09-15T21:29:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T21:29:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T21:29:24.281 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T21:29:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9. 32/37 2024-09-15T21:29:24.481 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/37 2024-09-15T21:29:24.508 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/37 2024-09-15T21:29:24.508 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:24.508 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:29:24.508 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:29:24.508 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:29:24.509 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:25.941 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 37/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 5/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9. 6/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T21:29:25.942 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T21:29:25.943 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T21:29:25.943 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T21:29:25.944 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T21:29:25.944 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T21:29:25.944 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T21:29:25.944 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T21:29:25.945 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T21:29:25.946 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T21:29:25.946 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T21:29:25.946 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T21:29:25.946 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T21:29:25.946 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T21:29:25.946 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T21:29:26.065 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/37 2024-09-15T21:29:26.094 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/37 2024-09-15T21:29:26.094 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:26.094 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:29:26.094 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:29:26.094 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:29:26.094 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:26.575 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 35/37 2024-09-15T21:29:26.600 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 35/37 2024-09-15T21:29:26.600 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:26.600 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:29:26.600 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:29:26.600 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:29:26.600 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:26.842 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 36/37 2024-09-15T21:29:26.866 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 36/37 2024-09-15T21:29:26.867 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:26.867 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:29:26.867 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:29:26.867 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:29:26.867 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:26.933 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 37/37 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:26.992 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:29:26.993 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:29:26.994 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:29:26.995 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:29:26.995 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:29:26.995 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:29:26.995 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:29:26.995 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:29:26.995 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:26.995 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:27.364 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-base 2024-09-15T21:29:27.929 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:54 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:28.019 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:29:28.060 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:28.061 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:29:28.061 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:28.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install cephadm 2024-09-15T21:29:28.677 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:55 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:28.741 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 37/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 5/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9. 6/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T21:29:28.742 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T21:29:28.743 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T21:29:28.744 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T21:29:28.745 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T21:29:28.807 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 784 k 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-15T21:29:28.808 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:28.809 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 784 k 2024-09-15T21:29:28.809 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 791 k 2024-09-15T21:29:28.809 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:29:29.095 INFO:teuthology.orchestra.run.smithi192.stdout:cephadm-19.3.0-4930.gfc3e0bff.el9.noarch.rpm 2.7 MB/s | 784 kB 00:00 2024-09-15T21:29:29.095 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:29.096 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.7 MB/s | 784 kB 00:00 2024-09-15T21:29:29.096 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:29:29.101 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:29:29.101 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:29:29.107 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:29:29.107 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:29:29.183 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:29:29.615 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:29:29.715 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:29:29.797 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T21:29:29.797 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:29.797 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:29:29.797 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:29.797 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:29.797 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:29.797 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:29:29.798 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:29:29.799 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:29.800 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:30.052 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:29:30.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-base 2024-09-15T21:29:30.305 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:29:30.306 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:30.306 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:29:30.306 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:30.306 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:30.306 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:30.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T21:29:30.745 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:56 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:30.838 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:29:30.883 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:29:30.884 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:30.945 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install cephadm 2024-09-15T21:29:31.076 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:00:58 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:31.209 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:31.209 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 145 k 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 145 k 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 432 k 2024-09-15T21:29:31.210 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:29:31.409 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-immutable-object-cache-19.3.0-4930.gfc3e0b 732 kB/s | 145 kB 00:00 2024-09-15T21:29:31.410 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:31.410 INFO:teuthology.orchestra.run.smithi192.stdout:Total 724 kB/s | 145 kB 00:00 2024-09-15T21:29:31.410 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:29:31.417 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:29:31.417 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:29:31.475 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:29:31.475 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:29:31.530 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:57 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:31.628 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:29:31.668 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:31.668 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 784 k 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 784 k 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 791 k 2024-09-15T21:29:31.669 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:29:31.719 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:29:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:29:31.744 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:31.744 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:29:31.744 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:31.941 INFO:teuthology.orchestra.run.smithi019.stdout:cephadm-19.3.0-4930.gfc3e0bff.el9.noarch.rpm 2.8 MB/s | 784 kB 00:00 2024-09-15T21:29:31.941 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:31.942 INFO:teuthology.orchestra.run.smithi019.stdout:Total 2.8 MB/s | 784 kB 00:00 2024-09-15T21:29:31.942 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:29:31.948 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:29:31.948 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:29:31.954 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:29:31.954 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:29:32.031 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:29:32.464 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:29:32.564 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:32.671 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:32.913 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr 2024-09-15T21:29:32.938 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:29:33.198 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:29:33.198 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:33.198 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:29:33.199 INFO:teuthology.orchestra.run.smithi019.stdout: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:33.199 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:33.199 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:33.399 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T21:29:33.480 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:00 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:33.575 INFO:teuthology.orchestra.run.smithi192.stdout:Package ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:29:33.617 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:33.618 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:29:33.618 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:33.706 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T21:29:33.993 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:00:59 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:34.132 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:34.132 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 145 k 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-09-15T21:29:34.133 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:34.134 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 145 k 2024-09-15T21:29:34.134 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 432 k 2024-09-15T21:29:34.134 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:29:34.268 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:01 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:34.338 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-immutable-object-cache-19.3.0-4930.gfc3e0b 708 kB/s | 145 kB 00:00 2024-09-15T21:29:34.339 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:34.339 INFO:teuthology.orchestra.run.smithi019.stdout:Total 702 kB/s | 145 kB 00:00 2024-09-15T21:29:34.339 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:29:34.347 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:29:34.347 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:29:34.401 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:34.402 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:34.402 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:29:34.402 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 14 M 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 34 k 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 16 k 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T21:29:34.403 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T21:29:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T21:29:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T21:29:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:34.404 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:29:34.404 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:34.404 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-09-15T21:29:34.404 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:34.405 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 19 M 2024-09-15T21:29:34.405 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 159 M 2024-09-15T21:29:34.405 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:29:34.408 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:29:34.408 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:29:34.554 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:29:34.631 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:29:34.658 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:29:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:29:34.658 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:29:34.658 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:35.271 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): ceph-prometheus-alerts-19.3.0-4930.gfc3 110 kB/s | 16 kB 00:00 2024-09-15T21:29:35.296 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): ceph-grafana-dashboards-19.3.0-4930.gfc 196 kB/s | 34 kB 00:00 2024-09-15T21:29:35.471 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 269 kB 00:00 2024-09-15T21:29:35.530 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-09-15T21:29:35.588 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 9.2 MB/s | 548 kB 00:00 2024-09-15T21:29:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:29:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:35.598 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:29:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:29:35.598 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:35.598 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:35.613 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-15T21:29:35.647 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 4.8 MB/s | 863 kB 00:00 2024-09-15T21:29:35.798 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): ceph-mgr-dashboard-19.3.0-4930.gfc3e0bf 21 MB/s | 14 MB 00:00 2024-09-15T21:29:35.817 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr 2024-09-15T21:29:35.840 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 754 kB/s | 144 kB 00:00 2024-09-15T21:29:35.882 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 377 kB/s | 31 kB 00:00 2024-09-15T21:29:35.907 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.8 MB/s | 188 kB 00:00 2024-09-15T21:29:35.974 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.7 MB/s | 2.0 MB 00:00 2024-09-15T21:29:35.978 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:35.978 INFO:teuthology.orchestra.run.smithi192.stdout:Total 12 MB/s | 19 MB 00:01 2024-09-15T21:29:36.029 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:29:36.049 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:29:36.049 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:29:36.319 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:29:36.319 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:29:36.412 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:02 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:36.507 INFO:teuthology.orchestra.run.smithi019.stdout:Package ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:29:36.551 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:36.552 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:29:36.553 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:36.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T21:29:36.933 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:29:37.008 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-15T21:29:37.051 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-15T21:29:37.097 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-15T21:29:37.137 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-15T21:29:37.211 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:03 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:37.223 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-15T21:29:37.350 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:37.351 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:29:37.351 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:37.351 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:29:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 14 M 2024-09-15T21:29:37.351 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:29:37.351 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 34 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 16 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:29:37.352 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:37.353 INFO:teuthology.orchestra.run.smithi019.stdout:Install 12 Packages 2024-09-15T21:29:37.353 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:37.353 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 19 M 2024-09-15T21:29:37.353 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 159 M 2024-09-15T21:29:37.353 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:29:37.354 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-15T21:29:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-15T21:29:37.517 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-15T21:29:37.545 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-15T21:29:37.577 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9 10/12 2024-09-15T21:29:37.926 INFO:teuthology.orchestra.run.smithi019.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4930.gfc 224 kB/s | 34 kB 00:00 2024-09-15T21:29:37.951 INFO:teuthology.orchestra.run.smithi019.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4930.gfc3 92 kB/s | 16 kB 00:00 2024-09-15T21:29:38.151 INFO:teuthology.orchestra.run.smithi019.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.3 MB/s | 269 kB 00:00 2024-09-15T21:29:38.285 INFO:teuthology.orchestra.run.smithi019.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-15T21:29:38.460 INFO:teuthology.orchestra.run.smithi019.stdout:(5/12): ceph-mgr-dashboard-19.3.0-4930.gfc3e0bf 21 MB/s | 14 MB 00:00 2024-09-15T21:29:38.485 INFO:teuthology.orchestra.run.smithi019.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.5 MB/s | 863 kB 00:00 2024-09-15T21:29:38.519 INFO:teuthology.orchestra.run.smithi019.stdout:(7/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.3 MB/s | 548 kB 00:00 2024-09-15T21:29:38.553 INFO:teuthology.orchestra.run.smithi019.stdout:(8/12): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-09-15T21:29:38.595 INFO:teuthology.orchestra.run.smithi019.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.2 MB/s | 144 kB 00:00 2024-09-15T21:29:38.620 INFO:teuthology.orchestra.run.smithi019.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 474 kB/s | 31 kB 00:00 2024-09-15T21:29:38.654 INFO:teuthology.orchestra.run.smithi019.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-09-15T21:29:38.770 INFO:teuthology.orchestra.run.smithi019.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 7.4 MB/s | 2.0 MB 00:00 2024-09-15T21:29:38.775 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:38.775 INFO:teuthology.orchestra.run.smithi019.stdout:Total 14 MB/s | 19 MB 00:01 2024-09-15T21:29:38.829 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:29:38.850 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:29:38.850 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:29:39.142 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:29:39.142 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:29:39.758 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:29:39.839 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-15T21:29:39.882 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-15T21:29:39.929 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-15T21:29:39.966 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-15T21:29:40.063 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-15T21:29:40.199 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-15T21:29:40.292 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-15T21:29:40.348 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-15T21:29:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-15T21:29:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9 10/12 2024-09-15T21:29:40.749 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el 11/12 2024-09-15T21:29:40.798 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 12/12 2024-09-15T21:29:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 12/12 2024-09-15T21:29:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el 1/12 2024-09-15T21:29:42.488 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 2/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9 3/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-15T21:29:42.489 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T21:29:42.821 INFO:teuthology.orchestra.run.smithi192.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T21:29:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T21:29:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T21:29:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T21:29:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T21:29:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T21:29:42.822 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:42.822 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:43.104 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T21:29:43.671 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:10 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:43.807 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 7.4 M 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T21:29:43.808 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout:======================================================================================= 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout:Install 12 Packages 2024-09-15T21:29:43.809 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:43.810 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 38 M 2024-09-15T21:29:43.810 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 212 M 2024-09-15T21:29:43.810 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:29:43.817 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el 11/12 2024-09-15T21:29:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 12/12 2024-09-15T21:29:44.280 INFO:teuthology.orchestra.run.smithi192.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 974 kB/s | 191 kB 00:00 2024-09-15T21:29:44.371 INFO:teuthology.orchestra.run.smithi192.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.7 MB/s | 801 kB 00:00 2024-09-15T21:29:44.547 INFO:teuthology.orchestra.run.smithi192.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-49 16 MB/s | 7.4 MB 00:00 2024-09-15T21:29:44.897 INFO:teuthology.orchestra.run.smithi192.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 53 kB/s | 33 kB 00:00 2024-09-15T21:29:45.097 INFO:teuthology.orchestra.run.smithi192.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 33 kB/s | 18 kB 00:00 2024-09-15T21:29:45.456 INFO:teuthology.orchestra.run.smithi192.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-15T21:29:45.481 INFO:teuthology.orchestra.run.smithi192.stdout:(7/12): openblas-0.3.26-2.el9.x86_64.rpm 66 kB/s | 38 kB 00:00 2024-09-15T21:29:45.707 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 12/12 2024-09-15T21:29:45.707 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el 1/12 2024-09-15T21:29:45.707 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 2/12 2024-09-15T21:29:45.707 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9 3/12 2024-09-15T21:29:45.707 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-15T21:29:45.707 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-15T21:29:45.708 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-15T21:29:45.708 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-15T21:29:45.708 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-15T21:29:45.708 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-15T21:29:45.708 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-15T21:29:45.708 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-15T21:29:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-15T21:29:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:46.030 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:29:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T21:29:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:46.031 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:46.215 INFO:teuthology.orchestra.run.smithi192.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 323 kB/s | 245 kB 00:00 2024-09-15T21:29:46.288 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T21:29:46.307 INFO:teuthology.orchestra.run.smithi192.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.1 MB/s | 4.9 MB 00:01 2024-09-15T21:29:46.674 INFO:teuthology.orchestra.run.smithi192.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.3 MB/s | 5.1 MB 00:01 2024-09-15T21:29:46.876 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:12 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:47.015 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 7.4 M 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T21:29:47.017 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout:======================================================================================= 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout:Install 12 Packages 2024-09-15T21:29:47.018 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:47.019 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 38 M 2024-09-15T21:29:47.019 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 212 M 2024-09-15T21:29:47.019 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:29:47.074 INFO:teuthology.orchestra.run.smithi192.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 292 kB/s | 250 kB 00:00 2024-09-15T21:29:47.527 INFO:teuthology.orchestra.run.smithi019.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2024-09-15T21:29:47.561 INFO:teuthology.orchestra.run.smithi019.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.2 MB/s | 801 kB 00:00 2024-09-15T21:29:47.636 INFO:teuthology.orchestra.run.smithi019.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 301 kB/s | 33 kB 00:00 2024-09-15T21:29:47.695 INFO:teuthology.orchestra.run.smithi019.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 312 kB/s | 18 kB 00:00 2024-09-15T21:29:47.745 INFO:teuthology.orchestra.run.smithi019.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 773 kB/s | 38 kB 00:00 2024-09-15T21:29:47.879 INFO:teuthology.orchestra.run.smithi019.stdout:(6/12): ceph-mgr-diskprediction-local-19.3.0-49 15 MB/s | 7.4 MB 00:00 2024-09-15T21:29:47.899 INFO:teuthology.orchestra.run.smithi192.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-09-15T21:29:47.901 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:47.901 INFO:teuthology.orchestra.run.smithi192.stdout:Total 9.3 MB/s | 38 MB 00:04 2024-09-15T21:29:47.937 INFO:teuthology.orchestra.run.smithi019.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.9 MB/s | 3.0 MB 00:00 2024-09-15T21:29:48.079 INFO:teuthology.orchestra.run.smithi019.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-15T21:29:48.129 INFO:teuthology.orchestra.run.smithi019.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.9 MB/s | 250 kB 00:00 2024-09-15T21:29:48.205 INFO:teuthology.orchestra.run.smithi019.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 752 kB/s | 245 kB 00:00 2024-09-15T21:29:48.262 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:29:48.288 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:29:48.289 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:29:48.438 INFO:teuthology.orchestra.run.smithi019.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 53 MB/s | 16 MB 00:00 2024-09-15T21:29:48.522 INFO:teuthology.orchestra.run.smithi019.stdout:(12/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 6.4 MB/s | 4.9 MB 00:00 2024-09-15T21:29:48.523 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:48.523 INFO:teuthology.orchestra.run.smithi019.stdout:Total 25 MB/s | 38 MB 00:01 2024-09-15T21:29:48.722 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:29:48.723 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:29:48.892 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:29:48.920 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:29:48.920 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:29:49.374 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:29:49.375 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:29:49.495 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:29:49.557 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T21:29:49.605 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T21:29:49.713 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T21:29:49.748 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T21:29:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T21:29:49.964 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T21:29:50.050 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T21:29:50.176 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:29:50.233 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T21:29:50.279 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T21:29:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T21:29:50.438 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T21:29:50.630 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T21:29:50.687 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T21:29:50.712 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T21:29:50.775 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T21:29:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T21:29:51.461 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T21:29:51.580 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T21:29:52.061 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T21:29:52.910 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T21:29:53.088 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T21:29:53.113 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 12/12 2024-09-15T21:29:53.992 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T21:29:54.028 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 12/12 2024-09-15T21:29:55.756 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 12/12 2024-09-15T21:29:55.756 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 1/12 2024-09-15T21:29:55.756 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T21:29:55.756 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T21:29:55.756 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T21:29:55.757 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T21:29:55.757 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T21:29:55.757 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T21:29:55.757 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T21:29:55.757 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T21:29:55.758 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T21:29:55.758 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T21:29:56.188 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T21:29:56.188 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:56.188 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:29:56.188 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:56.188 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:56.189 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:29:56.554 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-rook 2024-09-15T21:29:56.668 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 12/12 2024-09-15T21:29:56.668 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 1/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T21:29:56.669 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T21:29:57.123 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T21:29:57.124 INFO:teuthology.orchestra.run.smithi019.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T21:29:57.125 INFO:teuthology.orchestra.run.smithi019.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T21:29:57.125 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:57.125 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:29:57.138 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:24 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:29:57.275 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:29:57.276 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:57.276 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:29:57.276 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:57.276 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:29:57.276 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 49 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T21:29:57.277 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T21:29:57.278 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T21:29:57.278 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:57.278 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:29:57.278 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:29:57.278 INFO:teuthology.orchestra.run.smithi192.stdout:Install 13 Packages 2024-09-15T21:29:57.278 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:29:57.278 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 2.2 M 2024-09-15T21:29:57.279 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 26 M 2024-09-15T21:29:57.279 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:29:57.513 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-rook 2024-09-15T21:29:57.581 INFO:teuthology.orchestra.run.smithi192.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 216 kB/s | 26 kB 00:00 2024-09-15T21:29:57.606 INFO:teuthology.orchestra.run.smithi192.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 133 kB/s | 19 kB 00:00 2024-09-15T21:29:57.631 INFO:teuthology.orchestra.run.smithi192.stdout:(3/13): ceph-mgr-rook-19.3.0-4930.gfc3e0bff.el9 291 kB/s | 49 kB 00:00 2024-09-15T21:29:57.715 INFO:teuthology.orchestra.run.smithi192.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.0 MB/s | 222 kB 00:00 2024-09-15T21:29:57.740 INFO:teuthology.orchestra.run.smithi192.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-15T21:29:57.790 INFO:teuthology.orchestra.run.smithi192.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 648 kB/s | 32 kB 00:00 2024-09-15T21:29:57.816 INFO:teuthology.orchestra.run.smithi192.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 713 kB/s | 54 kB 00:00 2024-09-15T21:29:57.841 INFO:teuthology.orchestra.run.smithi192.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 281 kB/s | 14 kB 00:00 2024-09-15T21:29:57.874 INFO:teuthology.orchestra.run.smithi192.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-15T21:29:57.916 INFO:teuthology.orchestra.run.smithi192.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-15T21:29:57.942 INFO:teuthology.orchestra.run.smithi192.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 905 kB/s | 59 kB 00:00 2024-09-15T21:29:57.967 INFO:teuthology.orchestra.run.smithi192.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-15T21:29:58.025 INFO:teuthology.orchestra.run.smithi192.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.7 MB/s | 1.0 MB 00:00 2024-09-15T21:29:58.027 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:58.028 INFO:teuthology.orchestra.run.smithi192.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-09-15T21:29:58.090 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:29:58.105 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:29:58.105 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:29:58.112 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:24 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:29:58.193 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:29:58.194 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:29:58.253 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:29:58.254 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:58.254 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:29:58.254 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 49 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T21:29:58.255 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T21:29:58.256 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T21:29:58.256 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T21:29:58.256 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:58.256 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:29:58.256 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:29:58.256 INFO:teuthology.orchestra.run.smithi019.stdout:Install 13 Packages 2024-09-15T21:29:58.257 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:29:58.257 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 2.2 M 2024-09-15T21:29:58.257 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 26 M 2024-09-15T21:29:58.257 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:29:58.503 INFO:teuthology.orchestra.run.smithi019.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 235 kB/s | 19 kB 00:00 2024-09-15T21:29:58.529 INFO:teuthology.orchestra.run.smithi019.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 239 kB/s | 26 kB 00:00 2024-09-15T21:29:58.587 INFO:teuthology.orchestra.run.smithi019.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-09-15T21:29:58.613 INFO:teuthology.orchestra.run.smithi019.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-09-15T21:29:58.638 INFO:teuthology.orchestra.run.smithi019.stdout:(5/13): ceph-mgr-rook-19.3.0-4930.gfc3e0bff.el9 228 kB/s | 49 kB 00:00 2024-09-15T21:29:58.663 INFO:teuthology.orchestra.run.smithi019.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-15T21:29:58.688 INFO:teuthology.orchestra.run.smithi019.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.7 MB/s | 279 kB 00:00 2024-09-15T21:29:58.702 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:29:58.739 INFO:teuthology.orchestra.run.smithi019.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-15T21:29:58.764 INFO:teuthology.orchestra.run.smithi019.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 140 kB/s | 14 kB 00:00 2024-09-15T21:29:58.798 INFO:teuthology.orchestra.run.smithi019.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-09-15T21:29:58.814 INFO:teuthology.orchestra.run.smithi019.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-15T21:29:58.839 INFO:teuthology.orchestra.run.smithi019.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 2.1 MB/s | 90 kB 00:00 2024-09-15T21:29:58.871 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T21:29:58.881 INFO:teuthology.orchestra.run.smithi019.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.3 MB/s | 1.0 MB 00:00 2024-09-15T21:29:58.884 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:29:58.884 INFO:teuthology.orchestra.run.smithi019.stdout:Total 3.5 MB/s | 2.2 MB 00:00 2024-09-15T21:29:58.926 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T21:29:58.949 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:29:58.965 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:29:58.965 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:29:58.981 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T21:29:59.019 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T21:29:59.060 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T21:29:59.064 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:29:59.065 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:29:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T21:29:59.272 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T21:29:59.337 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T21:29:59.581 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:29:59.747 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T21:29:59.802 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T21:29:59.858 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T21:29:59.893 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T21:29:59.935 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T21:30:00.043 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T21:30:00.047 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T21:30:00.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T21:30:00.134 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T21:30:00.152 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T21:30:00.183 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T21:30:00.212 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 13/13 2024-09-15T21:30:00.213 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T21:30:00.982 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T21:30:01.013 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 13/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T21:30:01.014 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T21:30:01.025 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T21:30:01.059 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T21:30:01.108 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T21:30:01.136 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 13/13 2024-09-15T21:30:01.392 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T21:30:01.393 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T21:30:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T21:30:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T21:30:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T21:30:01.394 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:01.394 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:01.665 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T21:30:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 13/13 2024-09-15T21:30:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/13 2024-09-15T21:30:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T21:30:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T21:30:01.936 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T21:30:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T21:30:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T21:30:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T21:30:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T21:30:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T21:30:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T21:30:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T21:30:01.937 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T21:30:02.237 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:29 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T21:30:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:02.280 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:30:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T21:30:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T21:30:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T21:30:02.280 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:02.281 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:02.372 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:02.372 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 154 k 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T21:30:02.373 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:02.374 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:30:02.374 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:02.374 INFO:teuthology.orchestra.run.smithi192.stdout:Install 6 Packages 2024-09-15T21:30:02.374 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:02.374 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 7.0 M 2024-09-15T21:30:02.375 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 33 M 2024-09-15T21:30:02.375 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:30:02.547 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T21:30:03.001 INFO:teuthology.orchestra.run.smithi192.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-15T21:30:03.026 INFO:teuthology.orchestra.run.smithi192.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4930.gfc3e0bff.e 695 kB/s | 154 kB 00:00 2024-09-15T21:30:03.085 INFO:teuthology.orchestra.run.smithi192.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 990 kB/s | 58 kB 00:00 2024-09-15T21:30:03.119 INFO:teuthology.orchestra.run.smithi192.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-15T21:30:03.144 INFO:teuthology.orchestra.run.smithi192.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.8 MB/s | 548 kB 00:00 2024-09-15T21:30:03.147 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:29 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:03.236 INFO:teuthology.orchestra.run.smithi192.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-09-15T21:30:03.238 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:03.239 INFO:teuthology.orchestra.run.smithi192.stdout:Total 8.1 MB/s | 7.0 MB 00:00 2024-09-15T21:30:03.288 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:03.289 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:03.289 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:30:03.290 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:03.290 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:30:03.290 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 154 k 2024-09-15T21:30:03.290 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:30:03.290 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T21:30:03.290 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T21:30:03.290 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T21:30:03.290 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T21:30:03.291 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T21:30:03.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:03.291 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:30:03.291 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:03.291 INFO:teuthology.orchestra.run.smithi019.stdout:Install 6 Packages 2024-09-15T21:30:03.291 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:03.292 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 7.0 M 2024-09-15T21:30:03.292 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 33 M 2024-09-15T21:30:03.292 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:30:03.310 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:30:03.322 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:30:03.322 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:30:03.422 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:30:03.423 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:30:03.732 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:30:03.851 INFO:teuthology.orchestra.run.smithi019.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.7 MB/s | 248 kB 00:00 2024-09-15T21:30:03.896 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T21:30:03.918 INFO:teuthology.orchestra.run.smithi019.stdout:(2/6): ceph-mgr-cephadm-19.3.0-4930.gfc3e0bff.e 727 kB/s | 154 kB 00:00 2024-09-15T21:30:03.946 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T21:30:03.968 INFO:teuthology.orchestra.run.smithi019.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.6 MB/s | 548 kB 00:00 2024-09-15T21:30:03.995 INFO:teuthology.orchestra.run.smithi019.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 765 kB/s | 58 kB 00:00 2024-09-15T21:30:04.058 INFO:teuthology.orchestra.run.smithi019.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-15T21:30:04.102 INFO:teuthology.orchestra.run.smithi019.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-15T21:30:04.104 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:04.105 INFO:teuthology.orchestra.run.smithi019.stdout:Total 8.6 MB/s | 7.0 MB 00:00 2024-09-15T21:30:04.179 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:30:04.191 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:30:04.191 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:30:04.297 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:30:04.298 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:30:04.497 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T21:30:04.612 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T21:30:04.622 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:30:04.692 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T21:30:04.716 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 6/6 2024-09-15T21:30:04.792 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T21:30:04.838 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T21:30:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T21:30:05.493 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T21:30:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 6/6 2024-09-15T21:30:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/6 2024-09-15T21:30:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T21:30:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T21:30:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T21:30:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T21:30:05.569 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T21:30:05.608 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 6/6 2024-09-15T21:30:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T21:30:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:05.783 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:30:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T21:30:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T21:30:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T21:30:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T21:30:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T21:30:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:05.784 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:06.050 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-fuse 2024-09-15T21:30:06.383 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 6/6 2024-09-15T21:30:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/6 2024-09-15T21:30:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T21:30:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T21:30:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T21:30:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T21:30:06.617 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:33 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T21:30:06.678 INFO:teuthology.orchestra.run.smithi019.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T21:30:06.679 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:06.679 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:06.753 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:06.753 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:06.753 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:30:06.753 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 900 k 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout:Install 2 Packages 2024-09-15T21:30:06.754 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:06.755 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 980 k 2024-09-15T21:30:06.755 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 2.9 M 2024-09-15T21:30:06.755 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:30:06.934 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-fuse 2024-09-15T21:30:07.037 INFO:teuthology.orchestra.run.smithi192.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-09-15T21:30:07.171 INFO:teuthology.orchestra.run.smithi192.stdout:(2/2): ceph-fuse-19.3.0-4930.gfc3e0bff.el9.x86_ 3.0 MB/s | 900 kB 00:00 2024-09-15T21:30:07.172 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:07.172 INFO:teuthology.orchestra.run.smithi192.stdout:Total 2.3 MB/s | 980 kB 00:00 2024-09-15T21:30:07.183 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:30:07.204 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:30:07.204 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:30:07.284 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:30:07.284 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:30:07.475 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:30:07.522 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:33 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:07.615 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T21:30:07.662 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:07.662 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:07.662 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:30:07.664 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:07.664 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:30:07.664 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 900 k 2024-09-15T21:30:07.665 INFO:teuthology.orchestra.run.smithi019.stdout:Installing dependencies: 2024-09-15T21:30:07.665 INFO:teuthology.orchestra.run.smithi019.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:30:07.665 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:07.665 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:30:07.665 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:07.665 INFO:teuthology.orchestra.run.smithi019.stdout:Install 2 Packages 2024-09-15T21:30:07.665 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:07.666 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 980 k 2024-09-15T21:30:07.666 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 2.9 M 2024-09-15T21:30:07.666 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:30:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/2 2024-09-15T21:30:07.795 INFO:teuthology.orchestra.run.smithi019.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 729 kB/s | 80 kB 00:00 2024-09-15T21:30:07.937 INFO:teuthology.orchestra.run.smithi019.stdout:(2/2): ceph-fuse-19.3.0-4930.gfc3e0bff.el9.x86_ 3.5 MB/s | 900 kB 00:00 2024-09-15T21:30:07.938 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:07.938 INFO:teuthology.orchestra.run.smithi019.stdout:Total 3.5 MB/s | 980 kB 00:00 2024-09-15T21:30:07.950 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:30:07.973 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:30:07.973 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:30:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:30:08.058 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:30:08.255 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:30:08.403 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T21:30:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/2 2024-09-15T21:30:08.406 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/2 2024-09-15T21:30:08.473 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/2 2024-09-15T21:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T21:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T21:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:08.684 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:08.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-volume 2024-09-15T21:30:09.167 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/2 2024-09-15T21:30:09.167 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/2 2024-09-15T21:30:09.460 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T21:30:09.460 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:09.460 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:09.461 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T21:30:09.461 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:09.461 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:09.542 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:36 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:09.680 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:09.680 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:09.680 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 285 k 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:09.681 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 285 k 2024-09-15T21:30:09.682 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 1.3 M 2024-09-15T21:30:09.682 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:30:09.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install ceph-volume 2024-09-15T21:30:09.909 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-volume-19.3.0-4930.gfc3e0bff.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-15T21:30:09.909 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:09.910 INFO:teuthology.orchestra.run.smithi192.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-15T21:30:09.910 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:30:09.918 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:30:09.918 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:30:09.947 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:30:09.947 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:30:10.169 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:30:10.227 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:30:10.243 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:30:10.243 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:30:10.243 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T21:30:10.244 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:10.330 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:36 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:10.472 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repository Size 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 285 k 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 285 k 2024-09-15T21:30:10.473 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 1.3 M 2024-09-15T21:30:10.474 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:30:10.709 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-volume-19.3.0-4930.gfc3e0bff.el9.noarch.rp 1.2 MB/s | 285 kB 00:00 2024-09-15T21:30:10.710 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:10.710 INFO:teuthology.orchestra.run.smithi019.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-15T21:30:10.710 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:30:10.719 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:30:10.719 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:30:10.755 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:30:10.755 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:30:10.987 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:30:11.053 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:30:11.070 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:30:11.071 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:30:11.071 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T21:30:11.071 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:11.191 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:30:11.192 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:11.192 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:11.192 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:30:11.192 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:11.192 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:11.493 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados-devel 2024-09-15T21:30:12.082 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:30:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:12.179 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:30:12.179 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:12.179 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:12.222 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 127 k 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:12.223 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-15T21:30:12.224 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:12.224 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 127 k 2024-09-15T21:30:12.224 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 456 k 2024-09-15T21:30:12.224 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:30:12.439 INFO:teuthology.orchestra.run.smithi192.stdout:librados-devel-19.3.0-4930.gfc3e0bff.el9.x86_64 589 kB/s | 127 kB 00:00 2024-09-15T21:30:12.440 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:12.440 INFO:teuthology.orchestra.run.smithi192.stdout:Total 585 kB/s | 127 kB 00:00 2024-09-15T21:30:12.440 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:30:12.445 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:30:12.445 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:30:12.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados-devel 2024-09-15T21:30:12.509 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:30:12.510 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:30:12.666 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:30:12.762 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:13.101 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:39 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:13.181 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:13.240 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:13.241 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-09-15T21:30:13.241 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:13.241 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:30:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 127 k 2024-09-15T21:30:13.241 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:13.241 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:30:13.242 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:13.242 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-09-15T21:30:13.242 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:13.242 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 127 k 2024-09-15T21:30:13.242 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 456 k 2024-09-15T21:30:13.242 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:30:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:13.440 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:13.441 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:13.448 INFO:teuthology.orchestra.run.smithi019.stdout:librados-devel-19.3.0-4930.gfc3e0bff.el9.x86_64 614 kB/s | 127 kB 00:00 2024-09-15T21:30:13.448 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:13.449 INFO:teuthology.orchestra.run.smithi019.stdout:Total 610 kB/s | 127 kB 00:00 2024-09-15T21:30:13.449 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:30:13.455 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:30:13.455 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:30:13.523 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:30:13.523 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:30:13.659 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs2 2024-09-15T21:30:13.678 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:30:13.799 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:14.224 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:14.246 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:14.340 INFO:teuthology.orchestra.run.smithi192.stdout:Package libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:14.385 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:14.386 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:30:14.386 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:14.476 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:14.477 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:14.477 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:14.494 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install libcephfs-devel 2024-09-15T21:30:14.728 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs2 2024-09-15T21:30:15.063 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:15.196 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:15.196 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:15.196 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-15T21:30:15.197 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:15.197 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:30:15.197 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 32 k 2024-09-15T21:30:15.197 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:15.197 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:30:15.197 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:15.197 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-15T21:30:15.199 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:15.200 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 32 k 2024-09-15T21:30:15.200 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 142 k 2024-09-15T21:30:15.200 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:30:15.324 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:41 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:15.350 INFO:teuthology.orchestra.run.smithi192.stdout:libcephfs-devel-19.3.0-4930.gfc3e0bff.el9.x86_6 208 kB/s | 32 kB 00:00 2024-09-15T21:30:15.351 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:15.351 INFO:teuthology.orchestra.run.smithi192.stdout:Total 206 kB/s | 32 kB 00:00 2024-09-15T21:30:15.351 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:30:15.355 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:30:15.355 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:30:15.378 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:30:15.378 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:30:15.420 INFO:teuthology.orchestra.run.smithi019.stdout:Package libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:15.467 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:15.468 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:30:15.468 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:30:15.531 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install libcephfs-devel 2024-09-15T21:30:15.564 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:15.952 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:16.128 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:16.197 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:16.198 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:16.198 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:16.198 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:16.268 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:16.269 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-09-15T21:30:16.269 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:16.269 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:30:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 32 k 2024-09-15T21:30:16.269 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:16.269 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:30:16.269 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:16.270 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-09-15T21:30:16.270 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:16.270 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 32 k 2024-09-15T21:30:16.270 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 142 k 2024-09-15T21:30:16.270 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:30:16.377 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librados2 2024-09-15T21:30:16.419 INFO:teuthology.orchestra.run.smithi019.stdout:libcephfs-devel-19.3.0-4930.gfc3e0bff.el9.x86_6 213 kB/s | 32 kB 00:00 2024-09-15T21:30:16.419 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:16.419 INFO:teuthology.orchestra.run.smithi019.stdout:Total 211 kB/s | 32 kB 00:00 2024-09-15T21:30:16.420 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:30:16.423 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:30:16.423 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:30:16.444 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:30:16.444 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:30:16.538 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:30:16.645 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:16.946 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:17.016 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:17.036 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:17.080 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:17.081 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:30:17.081 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:17.154 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install librbd1 2024-09-15T21:30:17.267 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:17.267 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:17.267 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:17.267 INFO:teuthology.orchestra.run.smithi019.stdout: libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:17.267 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:17.267 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:17.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librados2 2024-09-15T21:30:17.723 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:17.814 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:17.858 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:17.859 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:30:17.859 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:17.925 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rados 2024-09-15T21:30:18.022 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:18.116 INFO:teuthology.orchestra.run.smithi019.stdout:Package librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:18.162 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:18.163 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:30:18.163 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:18.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install librbd1 2024-09-15T21:30:18.495 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:18.586 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:18.629 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:18.630 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:30:18.630 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:18.703 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rgw 2024-09-15T21:30:18.819 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:44 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:18.914 INFO:teuthology.orchestra.run.smithi019.stdout:Package librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:18.960 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:18.961 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:30:18.961 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:19.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rados 2024-09-15T21:30:19.269 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:19.360 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:19.402 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:19.403 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:30:19.404 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:19.482 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-cephfs 2024-09-15T21:30:19.611 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:19.705 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:19.751 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:19.752 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:30:19.752 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:19.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rgw 2024-09-15T21:30:20.053 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:20.147 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:20.189 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:20.190 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:30:20.190 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:20.255 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install python3-rbd 2024-09-15T21:30:20.407 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:20.501 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:20.547 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:30:20.549 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:20.660 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-cephfs 2024-09-15T21:30:20.825 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:20.916 INFO:teuthology.orchestra.run.smithi192.stdout:Package python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:20.959 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:20.960 INFO:teuthology.orchestra.run.smithi192.stdout:Nothing to do. 2024-09-15T21:30:20.960 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:21.044 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-fuse 2024-09-15T21:30:21.255 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:21.349 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:21.394 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:21.395 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:30:21.395 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:21.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install python3-rbd 2024-09-15T21:30:21.609 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:48 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:21.743 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:21.744 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-15T21:30:21.744 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:21.744 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:30:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 85 k 2024-09-15T21:30:21.744 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:21.745 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:30:21.745 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:21.745 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-15T21:30:21.745 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:21.745 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 85 k 2024-09-15T21:30:21.745 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 231 k 2024-09-15T21:30:21.745 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:30:21.953 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-fuse-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 410 kB/s | 85 kB 00:00 2024-09-15T21:30:21.954 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:21.954 INFO:teuthology.orchestra.run.smithi192.stdout:Total 408 kB/s | 85 kB 00:00 2024-09-15T21:30:21.955 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:30:21.961 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:30:21.961 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:30:22.025 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:30:22.025 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:30:22.057 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:48 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:22.151 INFO:teuthology.orchestra.run.smithi019.stdout:Package python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:30:22.179 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:30:22.197 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:22.198 INFO:teuthology.orchestra.run.smithi019.stdout:Nothing to do. 2024-09-15T21:30:22.198 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:22.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-fuse 2024-09-15T21:30:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:22.707 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:22.856 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:48 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:22.973 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:22.973 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:22.974 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:22.996 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 85 k 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:22.997 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 85 k 2024-09-15T21:30:22.998 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 231 k 2024-09-15T21:30:22.998 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:30:23.197 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-fuse-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 428 kB/s | 85 kB 00:00 2024-09-15T21:30:23.197 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:23.198 INFO:teuthology.orchestra.run.smithi019.stdout:Total 426 kB/s | 85 kB 00:00 2024-09-15T21:30:23.198 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:30:23.205 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:30:23.205 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:30:23.250 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-mirror 2024-09-15T21:30:23.273 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:30:23.273 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:30:23.425 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:30:23.531 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:23.848 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:50 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:23.916 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:23.991 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repo Size 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.2 M 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-15T21:30:23.992 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:23.993 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 3.2 M 2024-09-15T21:30:23.993 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 14 M 2024-09-15T21:30:23.993 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:30:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:24.192 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:24.192 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:24.323 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-mirror-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 9.7 MB/s | 3.2 MB 00:00 2024-09-15T21:30:24.324 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:24.324 INFO:teuthology.orchestra.run.smithi192.stdout:Total 9.6 MB/s | 3.2 MB 00:00 2024-09-15T21:30:24.325 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:30:24.334 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:30:24.334 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:30:24.418 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:30:24.419 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:30:24.454 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-mirror 2024-09-15T21:30:24.880 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:30:24.949 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:24.975 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:24.975 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:30:24.975 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:30:24.976 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:30:24.976 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:30:24.976 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:25.077 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:51 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:25.220 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:25.220 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:25.220 INFO:teuthology.orchestra.run.smithi019.stdout: Package Arch Version Repo Size 2024-09-15T21:30:25.221 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:25.221 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:30:25.221 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.2 M 2024-09-15T21:30:25.221 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:25.221 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:30:25.221 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:25.221 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-09-15T21:30:25.221 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:25.222 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 3.2 M 2024-09-15T21:30:25.222 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 14 M 2024-09-15T21:30:25.222 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:30:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-mirror-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 9.7 MB/s | 3.2 MB 00:00 2024-09-15T21:30:25.551 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:25.552 INFO:teuthology.orchestra.run.smithi019.stdout:Total 9.6 MB/s | 3.2 MB 00:00 2024-09-15T21:30:25.552 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:30:25.562 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:30:25.562 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:30:25.648 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:30:25.648 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:30:26.089 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:26.090 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:26.090 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:26.090 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:26.090 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:26.090 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:26.109 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:30:26.169 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:26.197 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:26.197 INFO:teuthology.orchestra.run.smithi019.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:30:26.197 INFO:teuthology.orchestra.run.smithi019.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:30:26.197 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:30:26.197 INFO:teuthology.orchestra.run.smithi019.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:30:26.197 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:26.325 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install rbd-nbd 2024-09-15T21:30:26.899 INFO:teuthology.orchestra.run.smithi192.stdout:Last metadata expiration check: 0:01:53 ago on Sun 15 Sep 2024 09:28:33 PM UTC. 2024-09-15T21:30:27.034 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: Package Architecture Version Repository Size 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 170 k 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout:Install 1 Package 2024-09-15T21:30:27.035 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 170 k 2024-09-15T21:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout:Installed size: 490 k 2024-09-15T21:30:27.036 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2024-09-15T21:30:27.252 INFO:teuthology.orchestra.run.smithi192.stdout:rbd-nbd-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 790 kB/s | 170 kB 00:00 2024-09-15T21:30:27.252 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:27.252 INFO:teuthology.orchestra.run.smithi192.stdout:Total 784 kB/s | 170 kB 00:00 2024-09-15T21:30:27.253 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2024-09-15T21:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2024-09-15T21:30:27.260 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2024-09-15T21:30:27.324 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2024-09-15T21:30:27.324 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2024-09-15T21:30:27.325 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:27.325 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:27.325 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:27.325 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:27.325 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:27.325 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:27.476 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2024-09-15T21:30:27.567 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:27.572 DEBUG:teuthology.orchestra.run.smithi019:> sudo yum -y install rbd-nbd 2024-09-15T21:30:27.987 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:28.165 INFO:teuthology.orchestra.run.smithi019.stdout:Last metadata expiration check: 0:01:54 ago on Sun 15 Sep 2024 09:28:34 PM UTC. 2024-09-15T21:30:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:28.248 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2024-09-15T21:30:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:28.248 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:28.248 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2024-09-15T21:30:28.307 INFO:teuthology.orchestra.run.smithi019.stdout:Dependencies resolved. 2024-09-15T21:30:28.308 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:28.308 INFO:teuthology.orchestra.run.smithi019.stdout: Package Architecture Version Repository Size 2024-09-15T21:30:28.308 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:28.308 INFO:teuthology.orchestra.run.smithi019.stdout:Installing: 2024-09-15T21:30:28.308 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 170 k 2024-09-15T21:30:28.308 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:28.308 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction Summary 2024-09-15T21:30:28.308 INFO:teuthology.orchestra.run.smithi019.stdout:================================================================================ 2024-09-15T21:30:28.309 INFO:teuthology.orchestra.run.smithi019.stdout:Install 1 Package 2024-09-15T21:30:28.309 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:28.309 INFO:teuthology.orchestra.run.smithi019.stdout:Total download size: 170 k 2024-09-15T21:30:28.309 INFO:teuthology.orchestra.run.smithi019.stdout:Installed size: 490 k 2024-09-15T21:30:28.309 INFO:teuthology.orchestra.run.smithi019.stdout:Downloading Packages: 2024-09-15T21:30:28.474 DEBUG:teuthology.parallel:result is None 2024-09-15T21:30:28.513 INFO:teuthology.orchestra.run.smithi019.stdout:rbd-nbd-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 836 kB/s | 170 kB 00:00 2024-09-15T21:30:28.513 INFO:teuthology.orchestra.run.smithi019.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:30:28.513 INFO:teuthology.orchestra.run.smithi019.stdout:Total 829 kB/s | 170 kB 00:00 2024-09-15T21:30:28.514 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction check 2024-09-15T21:30:28.522 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction check succeeded. 2024-09-15T21:30:28.522 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction test 2024-09-15T21:30:28.590 INFO:teuthology.orchestra.run.smithi019.stdout:Transaction test succeeded. 2024-09-15T21:30:28.591 INFO:teuthology.orchestra.run.smithi019.stdout:Running transaction 2024-09-15T21:30:28.757 INFO:teuthology.orchestra.run.smithi019.stdout: Preparing : 1/1 2024-09-15T21:30:28.864 INFO:teuthology.orchestra.run.smithi019.stdout: Installing : rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:29.541 INFO:teuthology.orchestra.run.smithi019.stdout: Verifying : rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:30:29.541 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:29.541 INFO:teuthology.orchestra.run.smithi019.stdout:Installed: 2024-09-15T21:30:29.541 INFO:teuthology.orchestra.run.smithi019.stdout: rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:30:29.541 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:29.541 INFO:teuthology.orchestra.run.smithi019.stdout:Complete! 2024-09-15T21:30:29.794 DEBUG:teuthology.parallel:result is None 2024-09-15T21:30:29.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:30:29.939 DEBUG:teuthology.orchestra.run.smithi019:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:30:29.961 INFO:teuthology.orchestra.run.smithi019.stdout:19.3.0-4930.gfc3e0bff.el9 2024-09-15T21:30:29.962 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930.gfc3e0bff.el9 2024-09-15T21:30:29.962 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930.gfc3e0bff is installed. 2024-09-15T21:30:29.964 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:30:30.104 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:30:30.129 INFO:teuthology.orchestra.run.smithi192.stdout:19.3.0-4930.gfc3e0bff.el9 2024-09-15T21:30:30.130 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930.gfc3e0bff.el9 2024-09-15T21:30:30.130 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930.gfc3e0bff is installed. 2024-09-15T21:30:30.132 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T21:30:30.132 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:30.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:30:30.159 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:30:30.160 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:30:30.200 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T21:30:30.201 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:30.201 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:30:30.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:30:30.293 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:30:30.293 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:30:30.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:30:30.389 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T21:30:30.389 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:30.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:30:30.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:30:30.483 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:30:30.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:30:30.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:30:30.577 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T21:30:30.577 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:30.577 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:30:30.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:30:30.669 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:30:30.669 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:30:30.695 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:30:30.761 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T21:30:30.845 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T21:30:30.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 777 /var/log/ceph 2024-09-15T21:30:30.848 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2024-09-15T21:30:30.872 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T21:30:30.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:30:30.915 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:30:30.939 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T21:30:30.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:30:30.981 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:30:31.015 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T21:30:31.016 INFO:tasks.ceph:config {'conf': {'global': {'lockdep': True, 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T21:30:31.016 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715', 'branch': 'main', 'description': 'rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-comp-zlib} rados tasks/mon_recovery validater/lockdep}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7905715', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'lockdep': True, 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 2855, 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'sleep_before_teardown': 0, 'subset': '17936/100000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7498493279982dfe87f00616198a5967475f6169', 'targets': {'smithi019.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKrAKUib0qGfdgL8H4yxZJEGEgYd319GFBz4DpVOZ5PlazZ2iS7ZDGOH+JUqC1l8J97fIgL/I8Ssvo3npqmbKfI=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEocJVGefvwGCDH9LbgyByRr4qAT7uWQEUjUWwN+GvY9pHrDJBkHeNbjJ4ZnjDvvda641wwAI77RzGjrnkPm1q8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'mon_recovery': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_20:00:16', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T21:30:31.016 DEBUG:teuthology.orchestra.run.smithi019:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:30:31.049 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:30:31.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:30:31.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:30:31.130 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:31.130 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:30:31.184 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:30:31.184 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_1 2024-09-15T21:30:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:30:31.240 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:30:31.240 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-15T21:30:31.240 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:30:31.240 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:30:31.240 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-15 21:30:26.460075961 +0000 2024-09-15T21:30:31.240 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-15 21:27:54.545494091 +0000 2024-09-15T21:30:31.240 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-15 21:27:54.545494091 +0000 2024-09-15T21:30:31.241 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-15 21:27:54.545494091 +0000 2024-09-15T21:30:31.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-15T21:30:31.304 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000198944 s, 2.6 MB/s 2024-09-15T21:30:31.305 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:30:31.363 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_2 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-15 21:30:26.460075961 +0000 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-15 21:27:54.894490506 +0000 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-15 21:27:54.894490506 +0000 2024-09-15T21:30:31.419 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-15 21:27:54.894490506 +0000 2024-09-15T21:30:31.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:30:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-15T21:30:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-15T21:30:31.483 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000172158 s, 3.0 MB/s 2024-09-15T21:30:31.484 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:30:31.541 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_3 2024-09-15T21:30:31.596 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:30:31.597 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:30:31.597 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-15T21:30:31.597 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:30:31.597 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:30:31.597 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-15 21:30:26.461075950 +0000 2024-09-15T21:30:31.597 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-15 21:27:55.229487063 +0000 2024-09-15T21:30:31.597 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-15 21:27:55.229487063 +0000 2024-09-15T21:30:31.597 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-15 21:27:55.229487063 +0000 2024-09-15T21:30:31.598 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:30:31.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-15T21:30:31.661 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-15T21:30:31.661 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.00021958 s, 2.3 MB/s 2024-09-15T21:30:31.662 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:30:31.720 DEBUG:teuthology.orchestra.run.smithi019:> stat /dev/vg_nvme/lv_4 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout:Access: 2024-09-15 21:30:26.461075950 +0000 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout:Modify: 2024-09-15 21:27:55.572483536 +0000 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout:Change: 2024-09-15 21:27:55.572483536 +0000 2024-09-15T21:30:31.777 INFO:teuthology.orchestra.run.smithi019.stdout: Birth: 2024-09-15 21:27:55.572483536 +0000 2024-09-15T21:30:31.778 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:30:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records in 2024-09-15T21:30:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:1+0 records out 2024-09-15T21:30:31.841 INFO:teuthology.orchestra.run.smithi019.stderr:512 bytes copied, 0.000178874 s, 2.9 MB/s 2024-09-15T21:30:31.842 DEBUG:teuthology.orchestra.run.smithi019:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:30:31.898 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:31.898 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:30:31.899 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:30:31.917 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:30:31.918 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2024-09-15T21:30:31.977 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:30:31.977 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:30:31.977 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-15T21:30:31.977 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:30:31.977 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:30:31.978 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-15 21:30:25.239880605 +0000 2024-09-15T21:30:31.978 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-15 21:27:54.531652612 +0000 2024-09-15T21:30:31.978 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-15 21:27:54.531652612 +0000 2024-09-15T21:30:31.978 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-15 21:27:54.531652612 +0000 2024-09-15T21:30:31.978 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:30:32.045 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-15T21:30:32.045 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-15T21:30:32.045 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000233238 s, 2.2 MB/s 2024-09-15T21:30:32.046 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:30:32.105 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2024-09-15T21:30:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:30:32.163 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:30:32.163 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T21:30:32.164 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:30:32.164 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:30:32.164 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-15 21:30:25.240880585 +0000 2024-09-15T21:30:32.164 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-15 21:27:54.853646005 +0000 2024-09-15T21:30:32.164 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-15 21:27:54.853646005 +0000 2024-09-15T21:30:32.164 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-15 21:27:54.853646005 +0000 2024-09-15T21:30:32.164 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:30:32.229 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-15T21:30:32.229 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-15T21:30:32.229 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000227507 s, 2.3 MB/s 2024-09-15T21:30:32.230 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:30:32.289 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2024-09-15T21:30:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:30:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:30:32.346 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T21:30:32.346 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:30:32.346 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:30:32.346 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-15 21:30:25.240880585 +0000 2024-09-15T21:30:32.346 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-15 21:27:55.174639419 +0000 2024-09-15T21:30:32.346 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-15 21:27:55.174639419 +0000 2024-09-15T21:30:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-15 21:27:55.174639419 +0000 2024-09-15T21:30:32.346 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:30:32.411 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-15T21:30:32.411 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-15T21:30:32.411 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000173419 s, 3.0 MB/s 2024-09-15T21:30:32.412 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:30:32.469 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2024-09-15 21:30:25.241880564 +0000 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2024-09-15 21:27:55.500632730 +0000 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2024-09-15 21:27:55.500632730 +0000 2024-09-15T21:30:32.525 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2024-09-15 21:27:55.500632730 +0000 2024-09-15T21:30:32.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:30:32.589 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2024-09-15T21:30:32.589 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2024-09-15T21:30:32.589 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000296055 s, 1.7 MB/s 2024-09-15T21:30:32.590 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:30:32.647 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:32.647 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T21:30:32.647 INFO:tasks.ceph:Generating config... 2024-09-15T21:30:32.648 INFO:tasks.ceph:[global] lockdep = True 2024-09-15T21:30:32.648 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-15T21:30:32.648 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-15T21:30:32.649 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-15T21:30:32.649 INFO:tasks.ceph:[global] ms type = async 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T21:30:32.649 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] bluestore compression algorithm = zlib 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] bluestore write v2 = False 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] debug monc = 20 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T21:30:32.650 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T21:30:32.651 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-15T21:30:32.652 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T21:30:32.652 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T21:30:32.706 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T21:30:32.709 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T21:30:32.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:30:32.780 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.19'), ('mon.c', '[v2:172.21.15.19:3301,v1:172.21.15.19:6790]'), ('mon.b', '172.21.15.192')] 2024-09-15T21:30:32.781 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],172.21.15.192', 'lockdep': True, 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-15T21:30:32.782 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:32.782 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:30:32.837 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.19 --addv c '[v2:172.21.15.19:3301,v1:172.21.15.19:6790]' --add b 172.21.15.192 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:32.925 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:32.925 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: generated fsid 13019d66-d4b6-4e2a-8322-38bcb4210bcf 2024-09-15T21:30:32.925 INFO:teuthology.orchestra.run.smithi019.stdout:setting min_mon_release = quincy 2024-09-15T21:30:32.925 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 0 2024-09-15T21:30:32.925 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 13019d66-d4b6-4e2a-8322-38bcb4210bcf 2024-09-15T21:30:32.926 INFO:teuthology.orchestra.run.smithi019.stdout:last_changed 2024-09-15T21:30:32.924046+0000 2024-09-15T21:30:32.926 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-09-15T21:30:32.924046+0000 2024-09-15T21:30:32.926 INFO:teuthology.orchestra.run.smithi019.stdout:min_mon_release 17 (quincy) 2024-09-15T21:30:32.926 INFO:teuthology.orchestra.run.smithi019.stdout:election_strategy: 1 2024-09-15T21:30:32.926 INFO:teuthology.orchestra.run.smithi019.stdout:0: [v2:172.21.15.19:3300/0,v1:172.21.15.19:6789/0] mon.a 2024-09-15T21:30:32.926 INFO:teuthology.orchestra.run.smithi019.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2024-09-15T21:30:32.926 INFO:teuthology.orchestra.run.smithi019.stdout:2: [v2:172.21.15.19:3301/0,v1:172.21.15.19:6790/0] mon.c 2024-09-15T21:30:32.926 INFO:teuthology.orchestra.run.smithi019.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T21:30:32.927 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:30:32.943 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 13019d66-d4b6-4e2a-8322-38bcb4210bcf... 2024-09-15T21:30:32.944 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:30:32.985 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:30:33.028 INFO:teuthology.orchestra.run.smithi019.stdout:[global] 2024-09-15T21:30:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: chdir = "" 2024-09-15T21:30:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:30:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: auth supported = cephx 2024-09-15T21:30:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: filestore xattr use omap = true 2024-09-15T21:30:33.028 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: mon clock drift allowed = 1.000 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: osd crush chooseleaf type = 0 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: auth debug = true 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on old message = true 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: ms die on bug = true 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: mon pg warn max object skew = 0 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.029 INFO:teuthology.orchestra.run.smithi019.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default size = 2 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow primary affinity = true 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd allow pg remap = true 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on no sortbitwise = false 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon warn on too few osds = false 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:30:33.030 INFO:teuthology.orchestra.run.smithi019.stdout: mon_allow_pool_size_one = true 2024-09-15T21:30:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:30:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: osd default data pool replay window = 5 2024-09-15T21:30:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: mon allow pool delete = true 2024-09-15T21:30:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.031 INFO:teuthology.orchestra.run.smithi019.stdout: mon cluster log file level = debug 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: debug asserts on shutdown = true 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: mon health detail to clog = false 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: mon host = "172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],172.21.15.192" 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: lockdep = True 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: mon client directed command retry = 5 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: mon election default strategy = 1 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: ms inject socket failures = 5000 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: ms type = async 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: fsid = 13019d66-d4b6-4e2a-8322-38bcb4210bcf 2024-09-15T21:30:33.032 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout:[osd] 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd journal size = 100 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub load threshold = 5.0 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd scrub max interval = 600 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd recover clone overlap = true 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd recovery max chunk = 1048576 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug shutdown = true 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug op order = true 2024-09-15T21:30:33.033 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify stray on activate = true 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug trim objects = true 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: osd open classes on start = true 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug pg log writeout = true 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: osd map max advance = 10 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: journal zero on create = true 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:30:33.034 INFO:teuthology.orchestra.run.smithi019.stdout: filestore apply finisher threads = 3 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: bdev debug aio = true 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug misdirected ops = true 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore block size = 96636764160 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore compression algorithm = zlib 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore compression mode = aggressive 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore fsck on mount = True 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore write v2 = False 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: bluestore zero block detection = True 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluefs = 20 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: debug bluestore = 20 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: debug monc = 20 2024-09-15T21:30:33.035 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: debug osd = 20 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: debug rocksdb = 10 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd full ratio = 0.9 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd blocked scrub grace period = 3600 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify cached snaps = True 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd debug verify missing on start = True 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd max markdown count = 1000 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd mclock override recovery settings = True 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd objectstore = bluestore 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd op queue = debug_random 2024-09-15T21:30:33.036 INFO:teuthology.orchestra.run.smithi019.stdout: osd op queue cut off = debug_random 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: osd shutdown pgref assert = True 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr] 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: debug mgr = 20 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: mgr/telemetry/nag = false 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout:[mon] 2024-09-15T21:30:33.037 INFO:teuthology.orchestra.run.smithi019.stdout: debug ms = 1 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: debug mon = 20 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: debug paxos = 20 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: debug auth = 20 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: mon data avail warn = 5 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: mon mgr mkfs grace = 240 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd reporter subtree level = osd 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: mon osd prime pg temp = true 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:30:33.038 INFO:teuthology.orchestra.run.smithi019.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: # 1m isn't quite enough 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon min osdmap epochs = 50 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune interval = 2 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune min = 15 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon osdmap full prune txsize = 2 2024-09-15T21:30:33.039 INFO:teuthology.orchestra.run.smithi019.stdout: mon scrub interval = 300 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout: paxos service trim min = 10 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout:[client] 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout: rgw cache enabled = true 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable ops log = true 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout: rgw enable usage log = true 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.a] 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.c] 2024-09-15T21:30:33.040 INFO:teuthology.orchestra.run.smithi019.stdout:[mon.b] 2024-09-15T21:30:33.042 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2024-09-15T21:30:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2024-09-15T21:30:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:30:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2024-09-15T21:30:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.042 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2024-09-15T21:30:33.043 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:30:33.044 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],172.21.15.192" 2024-09-15T21:30:33.045 INFO:teuthology.orchestra.run.smithi192.stdout: lockdep = True 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: mon client directed command retry = 5 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: mon election default strategy = 1 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: ms inject socket failures = 5000 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: ms type = async 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = 13019d66-d4b6-4e2a-8322-38bcb4210bcf 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.046 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug trim objects = true 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.047 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression algorithm = zlib 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore compression mode = aggressive 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2024-09-15T21:30:33.048 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore write v2 = False 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore zero block detection = True 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 20 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 20 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: debug monc = 20 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 10 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: osd blocked scrub grace period = 3600 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify cached snaps = True 2024-09-15T21:30:33.049 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify missing on start = True 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: osd max markdown count = 1000 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock override recovery settings = True 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = bluestore 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: osd op queue = debug_random 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: osd op queue cut off = debug_random 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: osd shutdown pgref assert = True 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-09-15T21:30:33.050 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2024-09-15T21:30:33.051 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.052 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: mon min osdmap epochs = 50 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: mon osdmap full prune interval = 2 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: mon osdmap full prune min = 15 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: mon osdmap full prune txsize = 2 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: mon scrub interval = 300 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: paxos service trim min = 10 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:30:33.053 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:30:33.054 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2024-09-15T21:30:33.054 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2024-09-15T21:30:33.054 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2024-09-15T21:30:33.054 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T21:30:33.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T21:30:33.139 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T21:30:33.139 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:33.139 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T21:30:33.155 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:33.155 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T21:30:33.210 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:30:33.211 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:33.211 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:30:33.211 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:30:33.286 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:30:33.287 DEBUG:teuthology.orchestra.run.smithi019:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:33.343 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:30:33.344 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:30:33.344 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:30:33.344 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:30:33.379 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:30:33.379 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:33.434 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T21:30:33.435 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T21:30:33.435 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T21:30:33.487 INFO:teuthology.orchestra.run.smithi019.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-15T21:30:33.490 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:30:33.550 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:30:33.553 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T21:30:33.553 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T21:30:33.553 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-15T21:30:33.596 INFO:teuthology.orchestra.run.smithi019.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-15T21:30:33.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-15T21:30:33.650 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-15T21:30:33.662 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T21:30:33.662 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi019.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-15T21:30:33.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T21:30:33.690 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:33.690 INFO:tasks.ceph:role: osd.0 2024-09-15T21:30:33.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:30:33.690 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:30:33.763 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:30:33.763 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:30:33.763 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:30:33.763 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:30:33.763 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:30:33.763 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:30:33.763 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:30:33.764 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:30:33.764 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:30:33.764 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:30:34.284 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-09-15T21:30:34.286 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-09-15T21:30:34.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T21:30:34.433 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T21:30:34.467 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T21:30:34.535 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:34.535 INFO:tasks.ceph:role: osd.1 2024-09-15T21:30:34.536 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:30:34.536 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:30:34.610 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:30:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:30:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:30:34.610 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:30:34.610 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:30:34.611 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:30:34.611 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:30:34.611 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:30:34.611 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:30:34.611 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:30:35.065 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-09-15T21:30:35.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-09-15T21:30:35.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T21:30:35.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T21:30:35.195 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T21:30:35.261 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:35.261 INFO:tasks.ceph:role: osd.2 2024-09-15T21:30:35.261 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:30:35.261 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:30:35.334 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:30:35.335 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:30:35.822 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-09-15T21:30:35.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-09-15T21:30:35.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T21:30:35.974 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T21:30:36.007 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-15T21:30:36.074 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:36.074 INFO:tasks.ceph:role: osd.3 2024-09-15T21:30:36.074 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:30:36.075 DEBUG:teuthology.orchestra.run.smithi019:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:30:36.150 INFO:teuthology.orchestra.run.smithi019.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:30:36.151 INFO:teuthology.orchestra.run.smithi019.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:30:36.681 INFO:teuthology.orchestra.run.smithi019.stdout:Discarding blocks...Done. 2024-09-15T21:30:36.683 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi019.front.sepia.ceph.com -o noatime 2024-09-15T21:30:36.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-15T21:30:36.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-15T21:30:36.836 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:36.933+0000 7f1d92c17640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T21:30:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:36.934+0000 7f1d92c17640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T21:30:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:36.934+0000 7f1d92c17640 -1 bdev(0x55d086437800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T21:30:36.937 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:36.934+0000 7f1d92c17640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T21:30:39.856 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T21:30:39.886 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:39.975+0000 7f9768287640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T21:30:39.978 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:39.975+0000 7f9768287640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T21:30:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:39.975+0000 7f9768287640 -1 bdev(0x55dccce4f800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T21:30:39.979 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:39.975+0000 7f9768287640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T21:30:43.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T21:30:43.169 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:43.262 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:43.259+0000 7f99bb218640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T21:30:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:43.259+0000 7f99bb218640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T21:30:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:43.259+0000 7f99bb218640 -1 bdev(0x557ae9281800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T21:30:43.263 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:43.259+0000 7f99bb218640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T21:30:46.433 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T21:30:46.464 DEBUG:teuthology.orchestra.run.smithi019:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:46.554+0000 7f8f887a6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-15T21:30:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:46.555+0000 7f8f887a6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-15T21:30:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:46.555+0000 7f8f887a6640 -1 bdev(0x5616ef85b800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-15T21:30:46.557 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:30:46.555+0000 7f8f887a6640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-15T21:30:49.473 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-15T21:30:49.505 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-15T21:30:49.536 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:49.536 INFO:tasks.ceph:role: osd.4 2024-09-15T21:30:49.537 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:30:49.537 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:30:49.613 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:30:49.613 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:30:49.613 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:30:49.613 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:30:49.613 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:30:49.613 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:30:49.613 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:30:49.613 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:30:49.614 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:30:49.614 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:30:50.118 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-15T21:30:50.121 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-15T21:30:50.121 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-15T21:30:50.269 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-15T21:30:50.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-15T21:30:50.371 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:50.371 INFO:tasks.ceph:role: osd.5 2024-09-15T21:30:50.372 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:30:50.372 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:30:50.445 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:30:50.445 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:30:50.445 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:30:50.446 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:30:50.446 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:30:50.446 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:30:50.446 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:30:50.446 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:30:50.446 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:30:50.446 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:30:50.905 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-15T21:30:50.907 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-15T21:30:50.908 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-15T21:30:50.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-15T21:30:51.059 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-15T21:30:51.125 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:51.125 INFO:tasks.ceph:role: osd.6 2024-09-15T21:30:51.125 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:30:51.125 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:30:51.197 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:30:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:30:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:30:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:30:51.197 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:30:51.197 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:30:51.197 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:30:51.197 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:30:51.198 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:30:51.198 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:30:51.697 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-15T21:30:51.699 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-15T21:30:51.699 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-15T21:30:51.819 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-15T21:30:51.848 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-15T21:30:51.914 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-15T21:30:51.914 INFO:tasks.ceph:role: osd.7 2024-09-15T21:30:51.914 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:30:51.914 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-15T21:30:51.986 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:30:51.987 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:30:52.526 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2024-09-15T21:30:52.528 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2024-09-15T21:30:52.528 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-15T21:30:52.651 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-15T21:30:52.682 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:52.782 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:52.779+0000 7f4b3d4fe640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-15T21:30:52.782 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:52.780+0000 7f4b3d4fe640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-15T21:30:52.782 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:52.780+0000 7f4b3d4fe640 -1 bdev(0x55be26557800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-15T21:30:52.782 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:52.780+0000 7f4b3d4fe640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-15T21:30:55.691 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-15T21:30:55.717 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:55.802 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:55.800+0000 7f90c2813640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-15T21:30:55.803 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:55.800+0000 7f90c2813640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-15T21:30:55.803 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:55.800+0000 7f90c2813640 -1 bdev(0x55b9e6703800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-15T21:30:55.803 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:55.800+0000 7f90c2813640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-15T21:30:58.958 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-15T21:30:58.986 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:30:59.075 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:59.072+0000 7f46aeeed640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-15T21:30:59.075 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:59.073+0000 7f46aeeed640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-15T21:30:59.075 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:59.073+0000 7f46aeeed640 -1 bdev(0x556ab4df3800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-15T21:30:59.075 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:30:59.073+0000 7f46aeeed640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-15T21:31:01.987 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-15T21:31:02.015 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:31:02.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:31:02.101+0000 7fe5e749c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-15T21:31:02.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:31:02.101+0000 7fe5e749c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-15T21:31:02.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:31:02.101+0000 7fe5e749c640 -1 bdev(0x55d9d5935800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-15T21:31:02.104 INFO:teuthology.orchestra.run.smithi192.stderr:2024-09-15T21:31:02.102+0000 7fe5e749c640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-15T21:31:05.021 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-15T21:31:05.050 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T21:31:05.050 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:05.050 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-15T21:31:05.081 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:05.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T21:31:05.151 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:05.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T21:31:05.221 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:05.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T21:31:05.289 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:05.289 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-15T21:31:05.356 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:05.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T21:31:05.387 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:05.388 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-15T21:31:05.456 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:05.456 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-15T21:31:05.524 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:05.524 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-15T21:31:05.592 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:05.592 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-15T21:31:05.659 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:05.659 DEBUG:teuthology.orchestra.run.smithi019:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-15T21:31:05.677 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:05.677 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-15T21:31:05.716 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T21:31:05.717 DEBUG:teuthology.orchestra.run.smithi019:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:31:05.720 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:31:05.766 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.y] 2024-09-15T21:31:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD5UedmdaH/HBAAR/IUNGNn62G5bEdsbUooXg== 2024-09-15T21:31:05.766 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.0] 2024-09-15T21:31:05.766 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD8UedmxvHFNxAASlgVs2fV9KQ2QpyuI+hpGQ== 2024-09-15T21:31:05.766 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.1] 2024-09-15T21:31:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD/UedmCuk9OhAA/ZxaVm+aD5ngJgj/ACFEmQ== 2024-09-15T21:31:05.767 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.2] 2024-09-15T21:31:05.767 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQADUudmtFOQDxAAoAD+FQL7e+Sm5CY3LY9dJA== 2024-09-15T21:31:05.767 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.3] 2024-09-15T21:31:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQAGUudm2eweIRAAkhwjMGwdO/mwjt/VjcH7yA== 2024-09-15T21:31:05.768 INFO:teuthology.orchestra.run.smithi019.stdout:[mgr.x] 2024-09-15T21:31:05.768 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD5UedmR0q+IBAAothfQO3aOxtCNzKikiNGpg== 2024-09-15T21:31:05.768 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.4] 2024-09-15T21:31:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQAMUudmfLSKLhAA3dSOID4MvBM7eVsQK8w6JA== 2024-09-15T21:31:05.769 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.5] 2024-09-15T21:31:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQAPUudm9UnELxAADKcSbV+fFbRo+bl9feJ96A== 2024-09-15T21:31:05.769 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.6] 2024-09-15T21:31:05.769 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQATUudmyz5oBBAAjKJdH+I06GR7KBYaTD0RCg== 2024-09-15T21:31:05.770 INFO:teuthology.orchestra.run.smithi019.stdout:[osd.7] 2024-09-15T21:31:05.770 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQAWUudmzmYfBhAA/cFb01BecowOztp45re+fQ== 2024-09-15T21:31:05.770 INFO:teuthology.orchestra.run.smithi019.stdout:[client.0] 2024-09-15T21:31:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD5Uedm0wx1IxAAlNkic14BCVzqaMbdG9C1ag== 2024-09-15T21:31:05.771 INFO:teuthology.orchestra.run.smithi019.stdout:[client.1] 2024-09-15T21:31:05.771 INFO:teuthology.orchestra.run.smithi019.stdout: key = AQD5UedmQdesJhAAb///hPwJ5U4VXeAECYlSxQ== 2024-09-15T21:31:05.782 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.y] 2024-09-15T21:31:05.782 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD5UedmdaH/HBAAR/IUNGNn62G5bEdsbUooXg== 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD8UedmxvHFNxAASlgVs2fV9KQ2QpyuI+hpGQ== 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD/UedmCuk9OhAA/ZxaVm+aD5ngJgj/ACFEmQ== 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQADUudmtFOQDxAAoAD+FQL7e+Sm5CY3LY9dJA== 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAGUudm2eweIRAAkhwjMGwdO/mwjt/VjcH7yA== 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD5UedmR0q+IBAAothfQO3aOxtCNzKikiNGpg== 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAMUudmfLSKLhAA3dSOID4MvBM7eVsQK8w6JA== 2024-09-15T21:31:05.783 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAPUudm9UnELxAADKcSbV+fFbRo+bl9feJ96A== 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.6] 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQATUudmyz5oBBAAjKJdH+I06GR7KBYaTD0RCg== 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.7] 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQAWUudmzmYfBhAA/cFb01BecowOztp45re+fQ== 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD5Uedm0wx1IxAAlNkic14BCVzqaMbdG9C1ag== 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2024-09-15T21:31:05.784 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQD5UedmQdesJhAAb///hPwJ5U4VXeAECYlSxQ== 2024-09-15T21:31:05.785 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:31:05.816 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:31:05.878 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:05.880 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:05.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:05.936 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:05.987 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:05.989 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.042 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.093 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:31:06.096 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:31:06.147 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.149 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.201 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.203 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.254 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.257 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.308 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.310 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:31:06.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T21:31:06.364 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T21:31:06.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T21:31:06.422 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-15T21:31:06.476 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T21:31:06.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T21:31:06.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:31:06.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T21:31:06.630 DEBUG:teuthology.orchestra.run.smithi019:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T21:31:06.695 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:31:06.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T21:31:06.818 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T21:31:06.845 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:31:06.943 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T21:31:06.967 DEBUG:teuthology.orchestra.run.smithi019:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:31:06.970 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:31:07.022 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T21:31:07.022 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:31:07.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:31:07.025 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:31:07.025 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:31:07.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:31:07.027 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:31:07.027 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:31:07.027 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:31:07.064 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:31:07.064 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T21:31:07.065 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-15T21:31:07.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-15T21:31:07.067 INFO:tasks.ceph.mgr.y:Started 2024-09-15T21:31:07.067 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T21:31:07.067 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T21:31:07.069 INFO:tasks.ceph.mgr.x:Started 2024-09-15T21:31:07.069 DEBUG:tasks.ceph:set 0 configs 2024-09-15T21:31:07.069 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T21:31:07.070 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T21:31:12.435 INFO:teuthology.orchestra.run.smithi019.stderr:adjusted tunables profile to default 2024-09-15T21:31:12.448 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T21:31:12.448 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T21:31:12.448 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:12.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T21:31:12.476 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:12.476 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T21:31:12.541 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:12.541 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T21:31:12.608 DEBUG:teuthology.orchestra.run.smithi019:> set -ex 2024-09-15T21:31:12.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-15T21:31:12.674 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:12.674 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-15T21:31:12.701 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:12.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-15T21:31:12.766 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:12.767 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-15T21:31:12.831 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2024-09-15T21:31:12.831 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-15T21:31:12.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new f7eb9035-e8c2-4d90-a198-2bed78d5a3b5 0 2024-09-15T21:31:13.118 INFO:teuthology.orchestra.run.smithi192.stdout:0 2024-09-15T21:31:13.129 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a 1 2024-09-15T21:31:13.309 INFO:teuthology.orchestra.run.smithi192.stdout:1 2024-09-15T21:31:13.320 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 180bae67-0f64-45a0-9a58-f4b8ff2b90a5 2 2024-09-15T21:31:13.501 INFO:teuthology.orchestra.run.smithi192.stdout:2 2024-09-15T21:31:13.513 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new d05cec44-e228-4297-98c8-53e2ad91b37c 3 2024-09-15T21:31:13.696 INFO:teuthology.orchestra.run.smithi192.stdout:3 2024-09-15T21:31:13.707 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 5d7097a2-8165-4885-a356-f8e67851be5d 4 2024-09-15T21:31:13.888 INFO:teuthology.orchestra.run.smithi192.stdout:4 2024-09-15T21:31:13.899 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f 5 2024-09-15T21:31:14.093 INFO:teuthology.orchestra.run.smithi192.stdout:5 2024-09-15T21:31:14.104 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612 6 2024-09-15T21:31:14.284 INFO:teuthology.orchestra.run.smithi192.stdout:6 2024-09-15T21:31:14.296 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new fb77f072-c3ae-4914-81bd-473d30016320 7 2024-09-15T21:31:14.503 INFO:teuthology.orchestra.run.smithi192.stdout:7 2024-09-15T21:31:14.515 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:31:14.515 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T21:31:14.517 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:31:14.517 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:31:14.518 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:31:14.520 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:31:14.520 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:31:14.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:31:14.522 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:31:14.522 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-15T21:31:14.522 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-15T21:31:14.524 INFO:tasks.ceph.osd.3:Started 2024-09-15T21:31:14.525 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-15T21:31:14.525 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-15T21:31:14.527 INFO:tasks.ceph.osd.4:Started 2024-09-15T21:31:14.527 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-15T21:31:14.527 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-15T21:31:14.529 INFO:tasks.ceph.osd.5:Started 2024-09-15T21:31:14.529 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-15T21:31:14.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-15T21:31:14.533 INFO:tasks.ceph.osd.6:Started 2024-09-15T21:31:14.534 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-15T21:31:14.534 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-15T21:31:14.537 INFO:tasks.ceph.osd.7:Started 2024-09-15T21:31:14.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:14.757 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":10,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:31:14.500069+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:31:14.500069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:31:14.768 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T21:31:14.768 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T21:31:15.575 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:31:15.738 INFO:teuthology.misc.health.smithi019.stdout: 2024-09-15T21:31:15.739 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":10,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:31:14.500069+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:31:14.500069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:31:15.748 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-15T21:31:15.749 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:15.746+0000 7f737ea83640 -1 Falling back to public interface 2024-09-15T21:31:15.767 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:15.765+0000 7efd5fa33640 -1 Falling back to public interface 2024-09-15T21:31:15.982 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:15.980+0000 7f9b968a5640 -1 Falling back to public interface 2024-09-15T21:31:15.998 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:15.996+0000 7fbee6a18640 -1 Falling back to public interface 2024-09-15T21:31:16.006 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:16.004+0000 7f3ad600f640 -1 Falling back to public interface 2024-09-15T21:31:16.010 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:16.008+0000 7f265c430640 -1 Falling back to public interface 2024-09-15T21:31:16.011 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:16.008+0000 7f1cc8919640 -1 Falling back to public interface 2024-09-15T21:31:16.281 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:16.278+0000 7f0545a1e640 -1 Falling back to public interface 2024-09-15T21:31:17.617 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-15T21:31:17.614+0000 7fc357a17640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T21:31:17.617 INFO:tasks.ceph.mgr.x.smithi192.stderr:2024-09-15T21:31:17.614+0000 7fc357a17640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T21:31:19.335 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:19.333+0000 7f737ea83640 -1 osd.3 0 log_to_monitors true 2024-09-15T21:31:19.359 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:19.356+0000 7efd5fa33640 -1 osd.6 0 log_to_monitors true 2024-09-15T21:31:19.546 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:19.544+0000 7f9b968a5640 -1 osd.2 0 log_to_monitors true 2024-09-15T21:31:19.835 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:19.833+0000 7f1cc8919640 -1 osd.0 0 log_to_monitors true 2024-09-15T21:31:20.061 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:20.059+0000 7f3ad600f640 -1 osd.7 0 log_to_monitors true 2024-09-15T21:31:20.098 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:20.096+0000 7fbee6a18640 -1 osd.1 0 log_to_monitors true 2024-09-15T21:31:20.301 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:20.299+0000 7f265c430640 -1 osd.4 0 log_to_monitors true 2024-09-15T21:31:20.360 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:20.358+0000 7f0545a1e640 -1 osd.5 0 log_to_monitors true 2024-09-15T21:31:20.629 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:20.627+0000 7f737aa12640 -1 osd.3 0 waiting for initial osdmap 2024-09-15T21:31:20.632 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:20.630+0000 7f7376839640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:31:20.633 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:20.631+0000 7efd5b9c2640 -1 osd.6 0 waiting for initial osdmap 2024-09-15T21:31:20.636 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:20.634+0000 7efd577e9640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:31:20.639 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:20.637+0000 7f9b92834640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T21:31:20.642 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:20.640+0000 7f9b8e65b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:31:21.642 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:21.640+0000 7f054145e640 -1 osd.5 0 waiting for initial osdmap 2024-09-15T21:31:21.645 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:21.643+0000 7f053d285640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:31:22.554 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:31:22.871 INFO:teuthology.misc.health.smithi019.stdout: 2024-09-15T21:31:22.872 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":14,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:31:22.627903+0000","last_up_change":"2024-09-15T21:31:22.627903+0000","last_in_change":"2024-09-15T21:31:14.500069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:31:22.622713+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6809","nonce":1831747042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6811","nonce":1831747042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6815","nonce":1831747042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6813","nonce":1831747042}]},"public_addr":"172.21.15.19:6809/1831747042","cluster_addr":"172.21.15.19:6811/1831747042","heartbeat_back_addr":"172.21.15.19:6815/1831747042","heartbeat_front_addr":"172.21.15.19:6813/1831747042","state":["exists","up"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6801","nonce":3658047863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6803","nonce":3658047863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6807","nonce":3658047863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6805","nonce":3658047863}]},"public_addr":"172.21.15.19:6801/3658047863","cluster_addr":"172.21.15.19:6803/3658047863","heartbeat_back_addr":"172.21.15.19:6807/3658047863","heartbeat_front_addr":"172.21.15.19:6805/3658047863","state":["exists","up"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6825","nonce":1071217311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6827","nonce":1071217311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6831","nonce":1071217311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6829","nonce":1071217311}]},"public_addr":"172.21.15.192:6825/1071217311","cluster_addr":"172.21.15.192:6827/1071217311","heartbeat_back_addr":"172.21.15.192:6831/1071217311","heartbeat_front_addr":"172.21.15.192:6829/1071217311","state":["exists","up"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6801","nonce":1069316293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6803","nonce":1069316293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6807","nonce":1069316293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6805","nonce":1069316293}]},"public_addr":"172.21.15.192:6801/1069316293","cluster_addr":"172.21.15.192:6803/1069316293","heartbeat_back_addr":"172.21.15.192:6807/1069316293","heartbeat_front_addr":"172.21.15.192:6805/1069316293","state":["exists","up"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.562583+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.363254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.381780+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:31:22.886 DEBUG:teuthology.misc:4 of 8 OSDs are up 2024-09-15T21:31:24.411 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:24.409+0000 7f3ad21a6640 -1 osd.7 0 waiting for initial osdmap 2024-09-15T21:31:24.423 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:24.421+0000 7f3ac97b1640 -1 osd.7 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:31:24.441 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:24.439+0000 7f2657972640 -1 osd.4 0 waiting for initial osdmap 2024-09-15T21:31:24.457 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:24.455+0000 7f264f791640 -1 osd.4 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:31:24.602 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:24.600+0000 7fbee2ab3640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T21:31:24.602 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:24.600+0000 7f1cc48a8640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T21:31:24.614 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:24.612+0000 7f1cbc6c7640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:31:24.619 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:24.617+0000 7fbeda0be640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:31:29.693 DEBUG:teuthology.orchestra.run.smithi019:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:31:30.001 INFO:teuthology.misc.health.smithi019.stdout: 2024-09-15T21:31:30.001 INFO:teuthology.misc.health.smithi019.stdout:{"epoch":18,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:31:27.417464+0000","last_up_change":"2024-09-15T21:31:25.414443+0000","last_in_change":"2024-09-15T21:31:14.500069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:31:22.622713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6825","nonce":368705522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6827","nonce":368705522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6831","nonce":368705522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6829","nonce":368705522}]},"public_addr":"172.21.15.19:6825/368705522","cluster_addr":"172.21.15.19:6827/368705522","heartbeat_back_addr":"172.21.15.19:6831/368705522","heartbeat_front_addr":"172.21.15.19:6829/368705522","state":["exists","up"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6817","nonce":3888640163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6819","nonce":3888640163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6823","nonce":3888640163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6821","nonce":3888640163}]},"public_addr":"172.21.15.19:6817/3888640163","cluster_addr":"172.21.15.19:6819/3888640163","heartbeat_back_addr":"172.21.15.19:6823/3888640163","heartbeat_front_addr":"172.21.15.19:6821/3888640163","state":["exists","up"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6809","nonce":1831747042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6811","nonce":1831747042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6815","nonce":1831747042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6813","nonce":1831747042}]},"public_addr":"172.21.15.19:6809/1831747042","cluster_addr":"172.21.15.19:6811/1831747042","heartbeat_back_addr":"172.21.15.19:6815/1831747042","heartbeat_front_addr":"172.21.15.19:6813/1831747042","state":["exists","up"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6801","nonce":3658047863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6803","nonce":3658047863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6807","nonce":3658047863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6805","nonce":3658047863}]},"public_addr":"172.21.15.19:6801/3658047863","cluster_addr":"172.21.15.19:6803/3658047863","heartbeat_back_addr":"172.21.15.19:6807/3658047863","heartbeat_front_addr":"172.21.15.19:6805/3658047863","state":["exists","up"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6817","nonce":1988554297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6819","nonce":1988554297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6823","nonce":1988554297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6821","nonce":1988554297}]},"public_addr":"172.21.15.192:6817/1988554297","cluster_addr":"172.21.15.192:6819/1988554297","heartbeat_back_addr":"172.21.15.192:6823/1988554297","heartbeat_front_addr":"172.21.15.192:6821/1988554297","state":["exists","up"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6825","nonce":1071217311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6827","nonce":1071217311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6831","nonce":1071217311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6829","nonce":1071217311}]},"public_addr":"172.21.15.192:6825/1071217311","cluster_addr":"172.21.15.192:6827/1071217311","heartbeat_back_addr":"172.21.15.192:6831/1071217311","heartbeat_front_addr":"172.21.15.192:6829/1071217311","state":["exists","up"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6801","nonce":1069316293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6803","nonce":1069316293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6807","nonce":1069316293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6805","nonce":1069316293}]},"public_addr":"172.21.15.192:6801/1069316293","cluster_addr":"172.21.15.192:6803/1069316293","heartbeat_back_addr":"172.21.15.192:6807/1069316293","heartbeat_front_addr":"172.21.15.192:6805/1069316293","state":["exists","up"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6809","nonce":3119270077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6811","nonce":3119270077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6815","nonce":3119270077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6813","nonce":3119270077}]},"public_addr":"172.21.15.192:6809/3119270077","cluster_addr":"172.21.15.192:6811/3119270077","heartbeat_back_addr":"172.21.15.192:6815/3119270077","heartbeat_front_addr":"172.21.15.192:6813/3119270077","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.795411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.139237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.562583+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.363254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.344635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.323050+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.381780+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.049941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:31:30.013 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-15T21:31:30.013 INFO:tasks.ceph:Creating RBD pool 2024-09-15T21:31:30.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T21:31:30.443 INFO:teuthology.orchestra.run.smithi019.stderr:pool 'rbd' created 2024-09-15T21:31:30.458 DEBUG:teuthology.orchestra.run.smithi019:> rbd --cluster ceph pool init rbd 2024-09-15T21:31:33.471 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T21:31:33.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-15T21:31:33.472 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T21:31:33.780 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:33.793 INFO:teuthology.orchestra.run.smithi019.stdout:[{"version":5,"timestamp":"2024-09-15T21:31:24.605086+0000","name":"","changes":[{"name":"osd.0/osd_mclock_max_capacity_iops_ssd","new_value":"6811.647374"}]}] 2024-09-15T21:31:33.793 INFO:tasks.ceph_manager:config epoch is 5 2024-09-15T21:31:33.793 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:31:33.793 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:31:33.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:31:34.143 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:34.153 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":1131950815},{"type":"v1","addr":"172.21.15.192:6833","nonce":1131950815}]},"active_addr":"172.21.15.192:6833/1131950815","active_change":"2024-09-15T21:31:16.600839+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4104,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":326334057}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":3136721100}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":1031645565}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":1792086882}]}]} 2024-09-15T21:31:34.158 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:31:34.158 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:31:34.158 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:31:34.462 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:34.462 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:31:33.452339+0000","last_up_change":"2024-09-15T21:31:25.414443+0000","last_in_change":"2024-09-15T21:31:14.500069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:31:22.622713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:31:30.313202+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6825","nonce":368705522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6827","nonce":368705522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6831","nonce":368705522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6829","nonce":368705522}]},"public_addr":"172.21.15.19:6825/368705522","cluster_addr":"172.21.15.19:6827/368705522","heartbeat_back_addr":"172.21.15.19:6831/368705522","heartbeat_front_addr":"172.21.15.19:6829/368705522","state":["exists","up"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6817","nonce":3888640163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6819","nonce":3888640163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6823","nonce":3888640163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6821","nonce":3888640163}]},"public_addr":"172.21.15.19:6817/3888640163","cluster_addr":"172.21.15.19:6819/3888640163","heartbeat_back_addr":"172.21.15.19:6823/3888640163","heartbeat_front_addr":"172.21.15.19:6821/3888640163","state":["exists","up"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6809","nonce":1831747042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6811","nonce":1831747042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6815","nonce":1831747042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6813","nonce":1831747042}]},"public_addr":"172.21.15.19:6809/1831747042","cluster_addr":"172.21.15.19:6811/1831747042","heartbeat_back_addr":"172.21.15.19:6815/1831747042","heartbeat_front_addr":"172.21.15.19:6813/1831747042","state":["exists","up"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6801","nonce":3658047863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6803","nonce":3658047863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6807","nonce":3658047863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6805","nonce":3658047863}]},"public_addr":"172.21.15.19:6801/3658047863","cluster_addr":"172.21.15.19:6803/3658047863","heartbeat_back_addr":"172.21.15.19:6807/3658047863","heartbeat_front_addr":"172.21.15.19:6805/3658047863","state":["exists","up"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6817","nonce":1988554297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6819","nonce":1988554297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6823","nonce":1988554297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6821","nonce":1988554297}]},"public_addr":"172.21.15.192:6817/1988554297","cluster_addr":"172.21.15.192:6819/1988554297","heartbeat_back_addr":"172.21.15.192:6823/1988554297","heartbeat_front_addr":"172.21.15.192:6821/1988554297","state":["exists","up"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6825","nonce":1071217311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6827","nonce":1071217311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6831","nonce":1071217311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6829","nonce":1071217311}]},"public_addr":"172.21.15.192:6825/1071217311","cluster_addr":"172.21.15.192:6827/1071217311","heartbeat_back_addr":"172.21.15.192:6831/1071217311","heartbeat_front_addr":"172.21.15.192:6829/1071217311","state":["exists","up"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6801","nonce":1069316293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6803","nonce":1069316293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6807","nonce":1069316293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6805","nonce":1069316293}]},"public_addr":"172.21.15.192:6801/1069316293","cluster_addr":"172.21.15.192:6803/1069316293","heartbeat_back_addr":"172.21.15.192:6807/1069316293","heartbeat_front_addr":"172.21.15.192:6805/1069316293","state":["exists","up"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6809","nonce":3119270077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6811","nonce":3119270077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6815","nonce":3119270077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6813","nonce":3119270077}]},"public_addr":"172.21.15.192:6809/3119270077","cluster_addr":"172.21.15.192:6811/3119270077","heartbeat_back_addr":"172.21.15.192:6815/3119270077","heartbeat_front_addr":"172.21.15.192:6813/3119270077","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.795411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.139237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.562583+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.363254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.344635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.323050+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.381780+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.049941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:31:34.475 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:31:34.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:31:34.774 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:34.774 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":22,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:31:33.452339+0000","last_up_change":"2024-09-15T21:31:25.414443+0000","last_in_change":"2024-09-15T21:31:14.500069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:31:22.622713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:31:30.313202+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6825","nonce":368705522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6827","nonce":368705522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6831","nonce":368705522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6829","nonce":368705522}]},"public_addr":"172.21.15.19:6825/368705522","cluster_addr":"172.21.15.19:6827/368705522","heartbeat_back_addr":"172.21.15.19:6831/368705522","heartbeat_front_addr":"172.21.15.19:6829/368705522","state":["exists","up"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6817","nonce":3888640163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6819","nonce":3888640163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6823","nonce":3888640163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6821","nonce":3888640163}]},"public_addr":"172.21.15.19:6817/3888640163","cluster_addr":"172.21.15.19:6819/3888640163","heartbeat_back_addr":"172.21.15.19:6823/3888640163","heartbeat_front_addr":"172.21.15.19:6821/3888640163","state":["exists","up"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6809","nonce":1831747042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6811","nonce":1831747042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6815","nonce":1831747042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6813","nonce":1831747042}]},"public_addr":"172.21.15.19:6809/1831747042","cluster_addr":"172.21.15.19:6811/1831747042","heartbeat_back_addr":"172.21.15.19:6815/1831747042","heartbeat_front_addr":"172.21.15.19:6813/1831747042","state":["exists","up"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6801","nonce":3658047863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6803","nonce":3658047863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6807","nonce":3658047863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6805","nonce":3658047863}]},"public_addr":"172.21.15.19:6801/3658047863","cluster_addr":"172.21.15.19:6803/3658047863","heartbeat_back_addr":"172.21.15.19:6807/3658047863","heartbeat_front_addr":"172.21.15.19:6805/3658047863","state":["exists","up"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6817","nonce":1988554297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6819","nonce":1988554297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6823","nonce":1988554297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6821","nonce":1988554297}]},"public_addr":"172.21.15.192:6817/1988554297","cluster_addr":"172.21.15.192:6819/1988554297","heartbeat_back_addr":"172.21.15.192:6823/1988554297","heartbeat_front_addr":"172.21.15.192:6821/1988554297","state":["exists","up"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6825","nonce":1071217311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6827","nonce":1071217311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6831","nonce":1071217311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6829","nonce":1071217311}]},"public_addr":"172.21.15.192:6825/1071217311","cluster_addr":"172.21.15.192:6827/1071217311","heartbeat_back_addr":"172.21.15.192:6831/1071217311","heartbeat_front_addr":"172.21.15.192:6829/1071217311","state":["exists","up"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6801","nonce":1069316293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6803","nonce":1069316293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6807","nonce":1069316293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6805","nonce":1069316293}]},"public_addr":"172.21.15.192:6801/1069316293","cluster_addr":"172.21.15.192:6803/1069316293","heartbeat_back_addr":"172.21.15.192:6807/1069316293","heartbeat_front_addr":"172.21.15.192:6805/1069316293","state":["exists","up"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6809","nonce":3119270077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6811","nonce":3119270077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6815","nonce":3119270077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6813","nonce":3119270077}]},"public_addr":"172.21.15.192:6809/3119270077","cluster_addr":"172.21.15.192:6811/3119270077","heartbeat_back_addr":"172.21.15.192:6815/3119270077","heartbeat_front_addr":"172.21.15.192:6813/3119270077","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.795411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.139237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.562583+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.363254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.344635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.323050+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.381780+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.049941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:31:34.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:31:34.787 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:31:34.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:31:34.788 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:31:34.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T21:31:34.789 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T21:31:34.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T21:31:34.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T21:31:35.044 INFO:teuthology.orchestra.run.smithi019.stdout:60129542150 2024-09-15T21:31:35.045 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2024-09-15T21:31:35.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:31:35.046 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:31:35.047 INFO:teuthology.orchestra.run.smithi019.stdout:68719476742 2024-09-15T21:31:35.047 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:31:35.048 INFO:teuthology.orchestra.run.smithi019.stdout:55834574854 2024-09-15T21:31:35.048 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:31:35.049 INFO:teuthology.orchestra.run.smithi019.stdout:55834574853 2024-09-15T21:31:35.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:31:35.050 INFO:teuthology.orchestra.run.smithi019.stdout:68719476742 2024-09-15T21:31:35.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:31:35.051 INFO:teuthology.orchestra.run.smithi019.stdout:55834574854 2024-09-15T21:31:35.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:31:35.053 INFO:teuthology.orchestra.run.smithi019.stdout:68719476742 2024-09-15T21:31:35.053 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:31:35.712 INFO:teuthology.orchestra.run.smithi019.stdout:55834574853 2024-09-15T21:31:35.712 INFO:teuthology.orchestra.run.smithi019.stdout:60129542149 2024-09-15T21:31:35.716 INFO:teuthology.orchestra.run.smithi019.stdout:68719476740 2024-09-15T21:31:35.724 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2024-09-15T21:31:35.727 INFO:teuthology.orchestra.run.smithi019.stdout:55834574852 2024-09-15T21:31:35.734 INFO:teuthology.orchestra.run.smithi019.stdout:55834574853 2024-09-15T21:31:35.735 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-09-15T21:31:35.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-09-15T21:31:35.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476740 for osd.7 2024-09-15T21:31:35.741 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2024-09-15T21:31:35.743 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2024-09-15T21:31:35.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.4 2024-09-15T21:31:35.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-09-15T21:31:35.755 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-09-15T21:31:35.761 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.0 2024-09-15T21:31:35.762 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.1 2024-09-15T21:31:36.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:31:36.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:31:36.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:31:36.750 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:31:36.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:31:36.756 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:31:36.762 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:31:36.762 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:31:37.356 INFO:teuthology.orchestra.run.smithi019.stdout:55834574853 2024-09-15T21:31:37.377 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-09-15T21:31:37.393 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2024-09-15T21:31:37.407 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.1 2024-09-15T21:31:37.407 INFO:teuthology.orchestra.run.smithi019.stdout:55834574852 2024-09-15T21:31:37.412 INFO:teuthology.orchestra.run.smithi019.stdout:60129542149 2024-09-15T21:31:37.424 INFO:teuthology.orchestra.run.smithi019.stdout:55834574853 2024-09-15T21:31:37.427 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2024-09-15T21:31:37.429 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.5 2024-09-15T21:31:37.429 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2024-09-15T21:31:37.431 INFO:teuthology.orchestra.run.smithi019.stdout:68719476741 2024-09-15T21:31:37.441 INFO:teuthology.orchestra.run.smithi019.stdout:68719476740 2024-09-15T21:31:37.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-09-15T21:31:37.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.4 2024-09-15T21:31:37.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.0 2024-09-15T21:31:37.456 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476740 for osd.7 2024-09-15T21:31:38.378 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:31:38.408 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:31:38.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:31:38.430 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:31:38.446 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:31:38.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:31:38.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:31:38.457 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:31:38.961 INFO:teuthology.orchestra.run.smithi019.stdout:55834574855 2024-09-15T21:31:38.979 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.3 2024-09-15T21:31:38.979 DEBUG:teuthology.parallel:result is None 2024-09-15T21:31:39.030 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-09-15T21:31:39.034 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-09-15T21:31:39.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476743 for osd.0 2024-09-15T21:31:39.049 DEBUG:teuthology.parallel:result is None 2024-09-15T21:31:39.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476743 for osd.1 2024-09-15T21:31:39.050 DEBUG:teuthology.parallel:result is None 2024-09-15T21:31:39.069 INFO:teuthology.orchestra.run.smithi019.stdout:55834574854 2024-09-15T21:31:39.082 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.6 2024-09-15T21:31:39.082 DEBUG:teuthology.parallel:result is None 2024-09-15T21:31:39.103 INFO:teuthology.orchestra.run.smithi019.stdout:55834574855 2024-09-15T21:31:39.108 INFO:teuthology.orchestra.run.smithi019.stdout:68719476743 2024-09-15T21:31:39.116 INFO:teuthology.orchestra.run.smithi019.stdout:68719476742 2024-09-15T21:31:39.118 INFO:teuthology.orchestra.run.smithi019.stdout:60129542150 2024-09-15T21:31:39.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.2 2024-09-15T21:31:39.121 DEBUG:teuthology.parallel:result is None 2024-09-15T21:31:39.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476743 for osd.4 2024-09-15T21:31:39.124 DEBUG:teuthology.parallel:result is None 2024-09-15T21:31:39.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476741 got 68719476742 for osd.7 2024-09-15T21:31:39.132 DEBUG:teuthology.parallel:result is None 2024-09-15T21:31:39.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.5 2024-09-15T21:31:39.133 DEBUG:teuthology.parallel:result is None 2024-09-15T21:31:39.133 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:31:39.133 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:39.472 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:39.472 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:31:39.484 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-15T21:31:38.609566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218080,"kb_used_data":1808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756640,"statfs":{"total":773094113280,"available":772870799360,"internally_reserved":0,"allocated":1851392,"data_stored":1665462,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.161263"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602188+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602188+0000","last_peered":"2024-09-15T21:31:33.602188+0000","last_clean":"2024-09-15T21:31:33.602188+0000","last_became_active":"2024-09-15T21:31:31.455107+0000","last_became_peered":"2024-09-15T21:31:31.455107+0000","last_unstale":"2024-09-15T21:31:33.602188+0000","last_undegraded":"2024-09-15T21:31:33.602188+0000","last_fullsized":"2024-09-15T21:31:33.602188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:37:27.496347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465416+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:31:33.465416+0000","last_peered":"2024-09-15T21:31:33.465416+0000","last_clean":"2024-09-15T21:31:33.465416+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:31:33.465416+0000","last_undegraded":"2024-09-15T21:31:33.465416+0000","last_fullsized":"2024-09-15T21:31:33.465416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602227+0000","last_change":"2024-09-15T21:31:33.602470+0000","last_active":"2024-09-15T21:31:33.602227+0000","last_peered":"2024-09-15T21:31:33.602227+0000","last_clean":"2024-09-15T21:31:33.602227+0000","last_became_active":"2024-09-15T21:31:31.458361+0000","last_became_peered":"2024-09-15T21:31:31.458361+0000","last_unstale":"2024-09-15T21:31:33.602227+0000","last_undegraded":"2024-09-15T21:31:33.602227+0000","last_fullsized":"2024-09-15T21:31:33.602227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:47.228609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000522709,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465467+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:31:33.465467+0000","last_peered":"2024-09-15T21:31:33.465467+0000","last_clean":"2024-09-15T21:31:33.465467+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:31:33.465467+0000","last_undegraded":"2024-09-15T21:31:33.465467+0000","last_fullsized":"2024-09-15T21:31:33.465467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.464681+0000","last_change":"2024-09-15T21:31:33.463884+0000","last_active":"2024-09-15T21:31:33.464681+0000","last_peered":"2024-09-15T21:31:33.464681+0000","last_clean":"2024-09-15T21:31:33.464681+0000","last_became_active":"2024-09-15T21:31:31.458375+0000","last_became_peered":"2024-09-15T21:31:31.458375+0000","last_unstale":"2024-09-15T21:31:33.464681+0000","last_undegraded":"2024-09-15T21:31:33.464681+0000","last_fullsized":"2024-09-15T21:31:33.464681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:21.745019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154895,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.459144+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:31:33.459144+0000","last_peered":"2024-09-15T21:31:33.459144+0000","last_clean":"2024-09-15T21:31:33.459144+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:31:33.459144+0000","last_undegraded":"2024-09-15T21:31:33.459144+0000","last_fullsized":"2024-09-15T21:31:33.459144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602178+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602178+0000","last_peered":"2024-09-15T21:31:33.602178+0000","last_clean":"2024-09-15T21:31:33.602178+0000","last_became_active":"2024-09-15T21:31:31.458374+0000","last_became_peered":"2024-09-15T21:31:31.458374+0000","last_unstale":"2024-09-15T21:31:33.602178+0000","last_undegraded":"2024-09-15T21:31:33.602178+0000","last_fullsized":"2024-09-15T21:31:33.602178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:45.856320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049767300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.463323+0000","last_change":"2024-09-15T21:31:33.463323+0000","last_active":"2024-09-15T21:31:33.463323+0000","last_peered":"2024-09-15T21:31:33.463323+0000","last_clean":"2024-09-15T21:31:33.463323+0000","last_became_active":"2024-09-15T21:31:31.455121+0000","last_became_peered":"2024-09-15T21:31:31.455121+0000","last_unstale":"2024-09-15T21:31:33.463323+0000","last_undegraded":"2024-09-15T21:31:33.463323+0000","last_fullsized":"2024-09-15T21:31:33.463323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:01.950055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055004999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.601856+0000","last_change":"2024-09-15T21:31:26.424937+0000","last_active":"2024-09-15T21:31:33.601856+0000","last_peered":"2024-09-15T21:31:33.601856+0000","last_clean":"2024-09-15T21:31:33.601856+0000","last_became_active":"2024-09-15T21:31:26.424008+0000","last_became_peered":"2024-09-15T21:31:26.424008+0000","last_unstale":"2024-09-15T21:31:33.601856+0000","last_undegraded":"2024-09-15T21:31:33.601856+0000","last_fullsized":"2024-09-15T21:31:33.601856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:05.634396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476742,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:39.485 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:39.783 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:39.783 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:31:39.795 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-15T21:31:38.609566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218080,"kb_used_data":1808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756640,"statfs":{"total":773094113280,"available":772870799360,"internally_reserved":0,"allocated":1851392,"data_stored":1665462,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.161263"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602188+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602188+0000","last_peered":"2024-09-15T21:31:33.602188+0000","last_clean":"2024-09-15T21:31:33.602188+0000","last_became_active":"2024-09-15T21:31:31.455107+0000","last_became_peered":"2024-09-15T21:31:31.455107+0000","last_unstale":"2024-09-15T21:31:33.602188+0000","last_undegraded":"2024-09-15T21:31:33.602188+0000","last_fullsized":"2024-09-15T21:31:33.602188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:37:27.496347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465416+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:31:33.465416+0000","last_peered":"2024-09-15T21:31:33.465416+0000","last_clean":"2024-09-15T21:31:33.465416+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:31:33.465416+0000","last_undegraded":"2024-09-15T21:31:33.465416+0000","last_fullsized":"2024-09-15T21:31:33.465416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602227+0000","last_change":"2024-09-15T21:31:33.602470+0000","last_active":"2024-09-15T21:31:33.602227+0000","last_peered":"2024-09-15T21:31:33.602227+0000","last_clean":"2024-09-15T21:31:33.602227+0000","last_became_active":"2024-09-15T21:31:31.458361+0000","last_became_peered":"2024-09-15T21:31:31.458361+0000","last_unstale":"2024-09-15T21:31:33.602227+0000","last_undegraded":"2024-09-15T21:31:33.602227+0000","last_fullsized":"2024-09-15T21:31:33.602227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:47.228609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000522709,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465467+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:31:33.465467+0000","last_peered":"2024-09-15T21:31:33.465467+0000","last_clean":"2024-09-15T21:31:33.465467+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:31:33.465467+0000","last_undegraded":"2024-09-15T21:31:33.465467+0000","last_fullsized":"2024-09-15T21:31:33.465467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.464681+0000","last_change":"2024-09-15T21:31:33.463884+0000","last_active":"2024-09-15T21:31:33.464681+0000","last_peered":"2024-09-15T21:31:33.464681+0000","last_clean":"2024-09-15T21:31:33.464681+0000","last_became_active":"2024-09-15T21:31:31.458375+0000","last_became_peered":"2024-09-15T21:31:31.458375+0000","last_unstale":"2024-09-15T21:31:33.464681+0000","last_undegraded":"2024-09-15T21:31:33.464681+0000","last_fullsized":"2024-09-15T21:31:33.464681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:21.745019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154895,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.459144+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:31:33.459144+0000","last_peered":"2024-09-15T21:31:33.459144+0000","last_clean":"2024-09-15T21:31:33.459144+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:31:33.459144+0000","last_undegraded":"2024-09-15T21:31:33.459144+0000","last_fullsized":"2024-09-15T21:31:33.459144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602178+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602178+0000","last_peered":"2024-09-15T21:31:33.602178+0000","last_clean":"2024-09-15T21:31:33.602178+0000","last_became_active":"2024-09-15T21:31:31.458374+0000","last_became_peered":"2024-09-15T21:31:31.458374+0000","last_unstale":"2024-09-15T21:31:33.602178+0000","last_undegraded":"2024-09-15T21:31:33.602178+0000","last_fullsized":"2024-09-15T21:31:33.602178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:45.856320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049767300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.463323+0000","last_change":"2024-09-15T21:31:33.463323+0000","last_active":"2024-09-15T21:31:33.463323+0000","last_peered":"2024-09-15T21:31:33.463323+0000","last_clean":"2024-09-15T21:31:33.463323+0000","last_became_active":"2024-09-15T21:31:31.455121+0000","last_became_peered":"2024-09-15T21:31:31.455121+0000","last_unstale":"2024-09-15T21:31:33.463323+0000","last_undegraded":"2024-09-15T21:31:33.463323+0000","last_fullsized":"2024-09-15T21:31:33.463323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:01.950055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055004999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.601856+0000","last_change":"2024-09-15T21:31:26.424937+0000","last_active":"2024-09-15T21:31:33.601856+0000","last_peered":"2024-09-15T21:31:33.601856+0000","last_clean":"2024-09-15T21:31:33.601856+0000","last_became_active":"2024-09-15T21:31:26.424008+0000","last_became_peered":"2024-09-15T21:31:26.424008+0000","last_unstale":"2024-09-15T21:31:33.601856+0000","last_undegraded":"2024-09-15T21:31:33.601856+0000","last_fullsized":"2024-09-15T21:31:33.601856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:05.634396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476742,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:39.796 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:31:39.796 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:31:39.796 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:31:39.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:31:40.133 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:40.133 INFO:teuthology.orchestra.run.smithi019.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:31:40.146 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:31:40.146 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-15T21:31:40.157 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-15T21:31:40.157 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'lockdep': True, 'mon client directed command retry': 5, 'mon election default strategy': 1, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zlib', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2': False, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}} 2024-09-15T21:31:40.157 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-15T21:31:40.157 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-15T21:31:40.479 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-15T21:31:40.479 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:31:40.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:40.777 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:40.778 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:31:40.790 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-15T21:31:40.610105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218080,"kb_used_data":1808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756640,"statfs":{"total":773094113280,"available":772870799360,"internally_reserved":0,"allocated":1851392,"data_stored":1665462,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.156633"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602188+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602188+0000","last_peered":"2024-09-15T21:31:33.602188+0000","last_clean":"2024-09-15T21:31:33.602188+0000","last_became_active":"2024-09-15T21:31:31.455107+0000","last_became_peered":"2024-09-15T21:31:31.455107+0000","last_unstale":"2024-09-15T21:31:33.602188+0000","last_undegraded":"2024-09-15T21:31:33.602188+0000","last_fullsized":"2024-09-15T21:31:33.602188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:37:27.496347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465416+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:31:33.465416+0000","last_peered":"2024-09-15T21:31:33.465416+0000","last_clean":"2024-09-15T21:31:33.465416+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:31:33.465416+0000","last_undegraded":"2024-09-15T21:31:33.465416+0000","last_fullsized":"2024-09-15T21:31:33.465416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602227+0000","last_change":"2024-09-15T21:31:33.602470+0000","last_active":"2024-09-15T21:31:33.602227+0000","last_peered":"2024-09-15T21:31:33.602227+0000","last_clean":"2024-09-15T21:31:33.602227+0000","last_became_active":"2024-09-15T21:31:31.458361+0000","last_became_peered":"2024-09-15T21:31:31.458361+0000","last_unstale":"2024-09-15T21:31:33.602227+0000","last_undegraded":"2024-09-15T21:31:33.602227+0000","last_fullsized":"2024-09-15T21:31:33.602227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:47.228609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000522709,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465467+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:31:33.465467+0000","last_peered":"2024-09-15T21:31:33.465467+0000","last_clean":"2024-09-15T21:31:33.465467+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:31:33.465467+0000","last_undegraded":"2024-09-15T21:31:33.465467+0000","last_fullsized":"2024-09-15T21:31:33.465467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.464681+0000","last_change":"2024-09-15T21:31:33.463884+0000","last_active":"2024-09-15T21:31:33.464681+0000","last_peered":"2024-09-15T21:31:33.464681+0000","last_clean":"2024-09-15T21:31:33.464681+0000","last_became_active":"2024-09-15T21:31:31.458375+0000","last_became_peered":"2024-09-15T21:31:31.458375+0000","last_unstale":"2024-09-15T21:31:33.464681+0000","last_undegraded":"2024-09-15T21:31:33.464681+0000","last_fullsized":"2024-09-15T21:31:33.464681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:21.745019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154895,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.459144+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:31:33.459144+0000","last_peered":"2024-09-15T21:31:33.459144+0000","last_clean":"2024-09-15T21:31:33.459144+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:31:33.459144+0000","last_undegraded":"2024-09-15T21:31:33.459144+0000","last_fullsized":"2024-09-15T21:31:33.459144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602178+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602178+0000","last_peered":"2024-09-15T21:31:33.602178+0000","last_clean":"2024-09-15T21:31:33.602178+0000","last_became_active":"2024-09-15T21:31:31.458374+0000","last_became_peered":"2024-09-15T21:31:31.458374+0000","last_unstale":"2024-09-15T21:31:33.602178+0000","last_undegraded":"2024-09-15T21:31:33.602178+0000","last_fullsized":"2024-09-15T21:31:33.602178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:45.856320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049767300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.463323+0000","last_change":"2024-09-15T21:31:33.463323+0000","last_active":"2024-09-15T21:31:33.463323+0000","last_peered":"2024-09-15T21:31:33.463323+0000","last_clean":"2024-09-15T21:31:33.463323+0000","last_became_active":"2024-09-15T21:31:31.455121+0000","last_became_peered":"2024-09-15T21:31:31.455121+0000","last_unstale":"2024-09-15T21:31:33.463323+0000","last_undegraded":"2024-09-15T21:31:33.463323+0000","last_fullsized":"2024-09-15T21:31:33.463323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:01.950055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055004999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.601856+0000","last_change":"2024-09-15T21:31:26.424937+0000","last_active":"2024-09-15T21:31:33.601856+0000","last_peered":"2024-09-15T21:31:33.601856+0000","last_clean":"2024-09-15T21:31:33.601856+0000","last_became_active":"2024-09-15T21:31:26.424008+0000","last_became_peered":"2024-09-15T21:31:26.424008+0000","last_unstale":"2024-09-15T21:31:33.601856+0000","last_undegraded":"2024-09-15T21:31:33.601856+0000","last_fullsized":"2024-09-15T21:31:33.601856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:05.634396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476742,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:40.791 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:41.085 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:41.086 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:31:41.098 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-15T21:31:40.610105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218080,"kb_used_data":1808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756640,"statfs":{"total":773094113280,"available":772870799360,"internally_reserved":0,"allocated":1851392,"data_stored":1665462,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.156633"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602188+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602188+0000","last_peered":"2024-09-15T21:31:33.602188+0000","last_clean":"2024-09-15T21:31:33.602188+0000","last_became_active":"2024-09-15T21:31:31.455107+0000","last_became_peered":"2024-09-15T21:31:31.455107+0000","last_unstale":"2024-09-15T21:31:33.602188+0000","last_undegraded":"2024-09-15T21:31:33.602188+0000","last_fullsized":"2024-09-15T21:31:33.602188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:37:27.496347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465416+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:31:33.465416+0000","last_peered":"2024-09-15T21:31:33.465416+0000","last_clean":"2024-09-15T21:31:33.465416+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:31:33.465416+0000","last_undegraded":"2024-09-15T21:31:33.465416+0000","last_fullsized":"2024-09-15T21:31:33.465416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602227+0000","last_change":"2024-09-15T21:31:33.602470+0000","last_active":"2024-09-15T21:31:33.602227+0000","last_peered":"2024-09-15T21:31:33.602227+0000","last_clean":"2024-09-15T21:31:33.602227+0000","last_became_active":"2024-09-15T21:31:31.458361+0000","last_became_peered":"2024-09-15T21:31:31.458361+0000","last_unstale":"2024-09-15T21:31:33.602227+0000","last_undegraded":"2024-09-15T21:31:33.602227+0000","last_fullsized":"2024-09-15T21:31:33.602227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:47.228609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000522709,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465467+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:31:33.465467+0000","last_peered":"2024-09-15T21:31:33.465467+0000","last_clean":"2024-09-15T21:31:33.465467+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:31:33.465467+0000","last_undegraded":"2024-09-15T21:31:33.465467+0000","last_fullsized":"2024-09-15T21:31:33.465467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.464681+0000","last_change":"2024-09-15T21:31:33.463884+0000","last_active":"2024-09-15T21:31:33.464681+0000","last_peered":"2024-09-15T21:31:33.464681+0000","last_clean":"2024-09-15T21:31:33.464681+0000","last_became_active":"2024-09-15T21:31:31.458375+0000","last_became_peered":"2024-09-15T21:31:31.458375+0000","last_unstale":"2024-09-15T21:31:33.464681+0000","last_undegraded":"2024-09-15T21:31:33.464681+0000","last_fullsized":"2024-09-15T21:31:33.464681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:21.745019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154895,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.459144+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:31:33.459144+0000","last_peered":"2024-09-15T21:31:33.459144+0000","last_clean":"2024-09-15T21:31:33.459144+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:31:33.459144+0000","last_undegraded":"2024-09-15T21:31:33.459144+0000","last_fullsized":"2024-09-15T21:31:33.459144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602178+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602178+0000","last_peered":"2024-09-15T21:31:33.602178+0000","last_clean":"2024-09-15T21:31:33.602178+0000","last_became_active":"2024-09-15T21:31:31.458374+0000","last_became_peered":"2024-09-15T21:31:31.458374+0000","last_unstale":"2024-09-15T21:31:33.602178+0000","last_undegraded":"2024-09-15T21:31:33.602178+0000","last_fullsized":"2024-09-15T21:31:33.602178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:45.856320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049767300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.463323+0000","last_change":"2024-09-15T21:31:33.463323+0000","last_active":"2024-09-15T21:31:33.463323+0000","last_peered":"2024-09-15T21:31:33.463323+0000","last_clean":"2024-09-15T21:31:33.463323+0000","last_became_active":"2024-09-15T21:31:31.455121+0000","last_became_peered":"2024-09-15T21:31:31.455121+0000","last_unstale":"2024-09-15T21:31:33.463323+0000","last_undegraded":"2024-09-15T21:31:33.463323+0000","last_fullsized":"2024-09-15T21:31:33.463323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:01.950055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055004999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.601856+0000","last_change":"2024-09-15T21:31:26.424937+0000","last_active":"2024-09-15T21:31:33.601856+0000","last_peered":"2024-09-15T21:31:33.601856+0000","last_clean":"2024-09-15T21:31:33.601856+0000","last_became_active":"2024-09-15T21:31:26.424008+0000","last_became_peered":"2024-09-15T21:31:26.424008+0000","last_unstale":"2024-09-15T21:31:33.601856+0000","last_undegraded":"2024-09-15T21:31:33.601856+0000","last_fullsized":"2024-09-15T21:31:33.601856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:05.634396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476742,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:41.099 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:31:41.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T21:31:41.396 INFO:teuthology.orchestra.run.smithi019.stdout:epoch 22 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:fsid 13019d66-d4b6-4e2a-8322-38bcb4210bcf 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:created 2024-09-15T21:31:12.329884+0000 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:modified 2024-09-15T21:31:33.452339+0000 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:crush_version 5 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:full_ratio 0.95 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:backfillfull_ratio 0.9 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:nearfull_ratio 0.85 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:require_min_compat_client luminous 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:min_compat_client jewel 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:require_osd_release squid 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:stretch_mode_enabled false 2024-09-15T21:31:41.397 INFO:teuthology.orchestra.run.smithi019.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 18 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 22 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.99 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout: removed_snaps_queue [2~1] 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:max_osd 8 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6824/368705522,v1:172.21.15.19:6825/368705522] [v2:172.21.15.19:6826/368705522,v1:172.21.15.19:6827/368705522] exists,up f7eb9035-e8c2-4d90-a198-2bed78d5a3b5 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1 up in weight 1 up_from 16 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6816/3888640163,v1:172.21.15.19:6817/3888640163] [v2:172.21.15.19:6818/3888640163,v1:172.21.15.19:6819/3888640163] exists,up d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6808/1831747042,v1:172.21.15.19:6809/1831747042] [v2:172.21.15.19:6810/1831747042,v1:172.21.15.19:6811/1831747042] exists,up 180bae67-0f64-45a0-9a58-f4b8ff2b90a5 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6800/3658047863,v1:172.21.15.19:6801/3658047863] [v2:172.21.15.19:6802/3658047863,v1:172.21.15.19:6803/3658047863] exists,up d05cec44-e228-4297-98c8-53e2ad91b37c 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/1988554297,v1:172.21.15.192:6817/1988554297] [v2:172.21.15.192:6818/1988554297,v1:172.21.15.192:6819/1988554297] exists,up 5d7097a2-8165-4885-a356-f8e67851be5d 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/1071217311,v1:172.21.15.192:6825/1071217311] [v2:172.21.15.192:6826/1071217311,v1:172.21.15.192:6827/1071217311] exists,up b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/1069316293,v1:172.21.15.192:6801/1069316293] [v2:172.21.15.192:6802/1069316293,v1:172.21.15.192:6803/1069316293] exists,up 14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612 2024-09-15T21:31:41.398 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7 up in weight 1 up_from 16 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/3119270077,v1:172.21.15.192:6809/3119270077] [v2:172.21.15.192:6810/3119270077,v1:172.21.15.192:6811/3119270077] exists,up fb77f072-c3ae-4914-81bd-473d30016320 2024-09-15T21:31:41.409 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6824/368705522,v1:172.21.15.19:6825/368705522] [v2:172.21.15.19:6826/368705522,v1:172.21.15.19:6827/368705522] exists,up f7eb9035-e8c2-4d90-a198-2bed78d5a3b5', 'osd.1 up in weight 1 up_from 16 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6816/3888640163,v1:172.21.15.19:6817/3888640163] [v2:172.21.15.19:6818/3888640163,v1:172.21.15.19:6819/3888640163] exists,up d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a', 'osd.2 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6808/1831747042,v1:172.21.15.19:6809/1831747042] [v2:172.21.15.19:6810/1831747042,v1:172.21.15.19:6811/1831747042] exists,up 180bae67-0f64-45a0-9a58-f4b8ff2b90a5', 'osd.3 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.19:6800/3658047863,v1:172.21.15.19:6801/3658047863] [v2:172.21.15.19:6802/3658047863,v1:172.21.15.19:6803/3658047863] exists,up d05cec44-e228-4297-98c8-53e2ad91b37c', 'osd.4 up in weight 1 up_from 16 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6816/1988554297,v1:172.21.15.192:6817/1988554297] [v2:172.21.15.192:6818/1988554297,v1:172.21.15.192:6819/1988554297] exists,up 5d7097a2-8165-4885-a356-f8e67851be5d', 'osd.5 up in weight 1 up_from 14 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6824/1071217311,v1:172.21.15.192:6825/1071217311] [v2:172.21.15.192:6826/1071217311,v1:172.21.15.192:6827/1071217311] exists,up b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f', 'osd.6 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6800/1069316293,v1:172.21.15.192:6801/1069316293] [v2:172.21.15.192:6802/1069316293,v1:172.21.15.192:6803/1069316293] exists,up 14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612', 'osd.7 up in weight 1 up_from 16 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:172.21.15.192:6808/3119270077,v1:172.21.15.192:6809/3119270077] [v2:172.21.15.192:6810/3119270077,v1:172.21.15.192:6811/3119270077] exists,up fb77f072-c3ae-4914-81bd-473d30016320'] 2024-09-15T21:31:41.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-15T21:31:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "mon.a", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster": "ceph", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "admin_socket_mode": "", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_client_required": "cephx, none", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_cluster_required": "cephx", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_debug": "true", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-15T21:31:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_service_required": "cephx", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "auth_supported": "cephx", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_aio": "true", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_aio_poll_ms": "250", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_aio_reap_max": "16", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_async_discard_threads": "0", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_block_size": "4096", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_debug_aio": "false", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-15T21:31:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_debug_inflight_ios": "false", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_enable_discard": "false", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_flock_retry": "3", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_inject_crash": "0", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_ioring": "false", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_ioring_hipri": "false", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-15T21:31:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bdev_type": "", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_alloc_size": "1048576", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_allocator": "hybrid", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_buffered_io": "true", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_check_for_zeros": "false", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_compact_log_sync": "false", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-15T21:31:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_max_log_runway": "4194304", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_max_prefetch": "1048576", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_min_flush_size": "524288", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_min_log_runway": "1048576", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_replay_recovery": "false", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluefs_sync_write": "false", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_allocation_from_file": "true", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_allocator": "hybrid", 2024-09-15T21:31:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_async_db_compaction": "true", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bdev_label_multi": "true", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_create": "true", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_db_create": "false", 2024-09-15T21:31:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_db_path": "", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_db_size": "0", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_path": "", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_preallocate_file": "false", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_size": "107374182400", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_wal_create": "false", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_wal_path": "", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_block_wal_size": "100663296", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bluefs": "true", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-15T21:31:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_btree2_alloc_weight_factor": "2.000000", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_autotune": "true", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_size": "0", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-15T21:31:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cache_type": "2q", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_clone_cow": "true", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-15T21:31:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_mode": "none", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_csum_type": "crc32c", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_enforce_min_alloc_size": "0", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_freelist": "false", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-15T21:31:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_small_allocations": "0", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_default_buffered_read": "true", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_default_buffered_write": "false", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-15T21:31:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fail_eio": "false", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-15T21:31:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_on_mount": "false", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_on_umount": "false", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-15T21:31:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_ignore_data_csum": "false", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_log_op_age": "5.000000", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_max_alloc_size": "0", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_max_blob_size": "0", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_max_deferred_txc": "32", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_min_alloc_size": "0", 2024-09-15T21:31:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_nid_prealloc": "1024", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_retry_disk_reads": "3", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_rocksdb_cf": "true", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-15T21:31:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_spdk_mem": "512", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-15T21:31:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-15T21:31:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_tracing": "false", 2024-09-15T21:31:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-15T21:31:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-15T21:31:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-15T21:31:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_write_v2": "false", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_write_v2_random": "false", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "bluestore_zero_block_detection": "false", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-15T21:31:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephx_cluster_require_signatures": "false", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephx_cluster_require_version": "2", 2024-09-15T21:31:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "cephx_require_signatures": "false", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "cephx_require_version": "2", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "cephx_service_require_signatures": "false", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "cephx_service_require_version": "2", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "cephx_sign_messages": "true", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "chdir": "", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "client_acl_type": "", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "client_asio_thread_count": "2", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "client_cache_mid": "0.750000", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "client_cache_size": "16384", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "client_caps_release_delay": "5", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "client_check_pool_perm": "true", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-15T21:31:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "client_debug_force_sync_read": "false", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_debug_getattr_caps": "false", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_debug_inject_features": "", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_debug_inject_tick_delay": "0", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_die_on_failed_remount": "false", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_dirsize_rbytes": "true", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_force_lazyio": "false", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_fs": "", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_inject_release_failure": "false", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_max_inline_size": "4096", 2024-09-15T21:31:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_mds_namespace": "", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_metadata": "", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_mount_gid": "-1", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_mount_timeout": "300", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_mount_uid": "-1", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_mountpoint": "/", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_notify_timeout": "10", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_oc": "true", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_oc_max_dirty": "104857600", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_oc_max_objects": "1000", 2024-09-15T21:31:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "client_oc_size": "209715200", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_oc_target_dirty": "8388608", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_permissions": "true", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_quota": "true", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_quota_df": "true", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_readahead_max_bytes": "0", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_readahead_max_periods": "4", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_readahead_min": "131072", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_reconnect_stale": "false", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_shutdown_timeout": "30", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_snapdir": ".snap", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_tick_interval": "1", 2024-09-15T21:31:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "client_trace": "", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "client_try_dentry_invalidate": "false", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "client_use_faked_inos": "false", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "client_use_random_mds": "false", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "clog_to_graylog": "false", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "clog_to_graylog_port": "12201", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "clog_to_monitors": "default=true", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "clog_to_syslog": "false", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "clog_to_syslog_level": "info", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "-", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_network": "", 2024-09-15T21:31:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_network_interface": "", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "compressor_zlib_isal": "false", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "compressor_zlib_level": "5", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "compressor_zlib_winsize": "-15", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "compressor_zstd_level": "1", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crimson_alien_op_num_threads": "6", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crimson_osd_stat_interval": "0", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crimson_seastar_cpu_cores": "", 2024-09-15T21:31:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "crimson_seastar_num_threads": "0", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location_hook": "", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location_hook_timeout": "10", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "daemonize": "false", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "daos_pool": "tank", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_alienstore": "0/5", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_allow_any_pool_priority": "false", 2024-09-15T21:31:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_asok": "1/5", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_asok_assert_abort": "false", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_asserts_on_shutdown": "true", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_auth": "20/20", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_bdev": "1/3", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_bluefs": "1/5", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_bluestore": "1/5", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_buffer": "0/1", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_ceph_dedup": "0/5", 2024-09-15T21:31:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_ceph_exporter": "1/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_cephfs_mirror": "0/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_cephsqlite": "0/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_client": "0/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_compressor": "1/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_context": "0/1", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_crimson_interrupt": "0/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_crush": "1/1", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_crypto": "1/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_cyanstore": "0/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_deliberately_leak_memory": "false", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_disable_randomized_ping": "false", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_dpdk": "1/5", 2024-09-15T21:31:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_eventtrace": "1/5", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_filer": "0/1", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_filestore": "1/3", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_finisher": "1/1", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_fuse": "1/5", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_heartbeat_testing_span": "0", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_heartbeatmap": "1/5", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_javaclient": "1/5", 2024-09-15T21:31:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_journal": "1/3", 2024-09-15T21:31:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_journaler": "0/5", 2024-09-15T21:31:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_kstore": "1/5", 2024-09-15T21:31:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_lockdep": "0/1", 2024-09-15T21:31:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mclock": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mds": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mds_balancer": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mds_locker": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mds_log": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mds_log_expire": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mds_migrator": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mds_quiesce": "3/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_memstore": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mgr": "2/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mgrc": "1/5", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_mon": "20/20", 2024-09-15T21:31:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_monc": "0/10", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_ms": "1/1", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_none": "0/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_objclass": "0/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_objectcacher": "0/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_objecter": "0/1", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_optracker": "0/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_osd": "1/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_paxos": "20/20", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_perfcounter": "1/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_prioritycache": "1/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rados": "0/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rbd": "0/5", 2024-09-15T21:31:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rbd_mirror": "0/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rbd_pwl": "0/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rbd_replay": "0/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_refs": "0/0", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_reserver": "1/1", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rgw": "1/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rgw_access": "1/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rgw_datacache": "1/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rgw_dbstore": "1/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rgw_flight": "1/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rgw_notification": "1/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rgw_sync": "1/5", 2024-09-15T21:31:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_rocksdb": "4/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_backref": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_cache": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_cleaner": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_device": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_epm": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_journal": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_lba": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_odata": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_omap": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_onode": "0/5", 2024-09-15T21:31:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_t": "0/5", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_seastore_tm": "0/5", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_striper": "0/1", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_test": "0/5", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_throttle": "1/1", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_timer": "0/1", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_tp": "0/5", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "debug_trace": "1/5", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "device_failure_prediction_mode": "none", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-15T21:31:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "err_to_graylog": "false", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "err_to_journald": "false", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "err_to_stderr": "true", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "err_to_syslog": "false", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event_tracing": "false", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "exporter_addr": "0.0.0.0", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "exporter_cert_file": "", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "exporter_http_port": "9926", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "exporter_key_file": "", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "exporter_prio_limit": "5", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "exporter_sort_metrics": "true", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "exporter_stats_period": "5", 2024-09-15T21:31:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "fake_statfs_for_testing": "0", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "fatal_signal_handlers": "true", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filer_max_purge_ops": "10", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filer_max_truncate_ops": "128", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_apply_finisher_threads": "1", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_blackhole": "false", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_btrfs_clone_range": "true", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_btrfs_snap": "true", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_caller_concurrency": "10", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_collect_device_partition_information": "true", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_commit_timeout": "600.000000", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_debug_inject_read_err": "false", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_debug_omap_check": "false", 2024-09-15T21:31:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_debug_verify_split": "false", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_dump_file": "", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_fadvise": "true", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_fail_eio": "true", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_fd_cache_shards": "16", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_fd_cache_size": "128", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_fiemap": "false", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_fiemap_threshold": "4096", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_inject_stall": "0", 2024-09-15T21:31:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_journal_parallel": "false", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_journal_trailing": "false", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_journal_writeahead": "false", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_kill_at": "0", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_inline_xattrs": "0", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-15T21:31:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_xattr_value_size": "0", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_merge_threshold": "-10", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_odsync_write": "false", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_omap_backend": "rocksdb", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_omap_backend_path": "", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-15T21:31:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_op_thread_timeout": "60", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_op_threads": "2", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_punch_hole": "false", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_queue_max_ops": "50", 2024-09-15T21:31:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_seek_data_hole": "false", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_sloppy_crc": "false", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_splice": "false", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_split_multiple": "2", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_split_rand_factor": "20", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_update_to": "1000", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-15T21:31:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_enable": "true", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_xfs_extsize": "false", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "filestore_zfs_snap": "false", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "fio_dir": "/tmp/fio", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_allow_other": "true", 2024-09-15T21:31:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_atomic_o_trunc": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_big_writes": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_debug": "false", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_default_permissions": "false", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_disable_pagecache": "false", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_max_write": "0", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_multithreaded": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_require_active_mds": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_set_user_groups": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_splice_move": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_splice_read": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_splice_write": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-15T21:31:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "fuse_use_invalidate_cb": "true", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "gss_target_name": "ceph", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_file": "", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_inject_failure": "0", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_interval": "5", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "host": "smithi019", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-15T21:31:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "inject_early_sigterm": "false", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "jaeger_agent_port": "6799", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "jaeger_tracing_enable": "false", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_aio": "true", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_align_min_size": "65536", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_block_align": "true", 2024-09-15T21:31:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_block_size": "4096", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_dio": "true", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_discard": "false", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_force_aio": "false", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_ignore_corruption": "false", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_max_write_bytes": "10485760", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_max_write_entries": "100", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_replay_from": "0", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_write_header_frequency": "0", 2024-09-15T21:31:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "journal_zero_on_create": "false", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "journaler_prefetch_periods": "10", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "journaler_prezero_periods": "5", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "journaler_write_head_interval": "15", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "key": "", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "keyfile": "", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_backend": "rocksdb", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_default_stripe_size": "65536", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_fsck_on_mount": "false", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_max_bytes": "67108864", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_max_ops": "512", 2024-09-15T21:31:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_nid_prealloc": "1024", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_onode_map_size": "1024", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_sync_submit_transaction": "false", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "kstore_sync_transaction": "false", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "librados_thread_count": "2", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "lockdep": "true", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "lockdep_force_backtrace": "false", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_coarse_timestamps": "true", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_flush_on_exit": "false", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_graylog_host": "127.0.0.1", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_graylog_port": "12201", 2024-09-15T21:31:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "log_max_new": "1000", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_max_recent": "10000", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_stderr_prefix": "", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_stop_at_utilization": "0.970000", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_file": "true", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_graylog": "false", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_journald": "false", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_stderr": "false", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "log_to_syslog": "false", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "max_rotating_auth_attempts": "10", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_action_on_write_error": "1", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_alternate_name_max": "8192", 2024-09-15T21:31:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_asio_thread_count": "2", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_export_pin": "true", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_fragment_dirs": "true", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_fragment_interval": "5", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_interval": "10", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_max": "-1", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_max_until": "-1", 2024-09-15T21:31:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_merge_size": "50", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_midchunk": "0.300000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_min_start": "0.200000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_minchunk": "0.001000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_mode": "0", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_need_max": "1.200000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_need_min": "0.800000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_overload_epochs": "2", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_split_bits": "3", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-15T21:31:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_split_size": "10000", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_target_decay": "10.000000", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_beacon_grace": "15.000000", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_beacon_interval": "4.000000", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_mid": "0.700000", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_quiesce_delay": "0", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-15T21:31:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_release_free_interval": "10", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_reservation": "0.050000", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_trim_interval": "1", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cache_trim_threshold": "262144", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_client_prealloc_inos": "1000", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_connect_bootstrapping": "false", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_damage_table_max_entries": "10000", 2024-09-15T21:31:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_debug_auth_pins": "false", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_debug_frag": "false", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_debug_scatterstat": "false", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_debug_subtrees": "false", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_decay_halflife": "5.000000", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_default_dir_hash": "2", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_defer_session_stale": "true", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_deny_all_reconnect": "false", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dir_keys_per_op": "16384", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dir_max_commit_size": "10", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dir_max_entries": "0", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dir_prefetch": "true", 2024-09-15T21:31:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dump_cache_on_map": "false", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_early_reply": "true", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_enable_op_tracker": "true", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_enforce_unique_name": "true", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_export_ephemeral_random": "true", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-15T21:31:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_health_summarize_threshold": "10", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_inject_health_dummy": "false", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-15T21:31:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_inject_migrator_session_race": "false", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_join_fs": "", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_journal_format": "1", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_create_at": "0", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_dirfrag_at": "0", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_export_at": "0", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_import_at": "0", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_journal_at": "0", 2024-09-15T21:31:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_journal_expire_at": "0", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_journal_replay_at": "0", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_link_at": "0", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_mdstable_at": "0", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_openc_at": "0", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_rename_at": "0", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_kill_shutdown_at": "0", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_event_large_threshold": "524288", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_events_per_segment": "1024", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_max_events": "-1", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_max_segments": "128", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_pause": "false", 2024-09-15T21:31:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_segment_size": "0", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_trim_threshold": "128", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_log_warn_factor": "2.000000", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_caps_per_client": "1048576", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_completed_flushes": "100000", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_completed_requests": "100000", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_export_size": "20971520", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_file_recover": "32", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_purge_files": "64", 2024-09-15T21:31:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_purge_ops": "8192", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_max_snaps_per_dir": "100", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_metrics_update_interval": "2", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_min_caps_per_client": "100", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_min_caps_working_set": "10000", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_numa_node": "-1", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_op_complaint_time": "30.000000", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_op_history_duration": "600", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_op_history_size": "20", 2024-09-15T21:31:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_op_history_slow_op_size": "20", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_op_log_threshold": "5", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_ping_grace": "15", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_ping_interval": "5", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_recall_max_caps": "30000", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_recall_warning_threshold": "262144", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-15T21:31:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_replay_interval": "1.000000", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_root_ino_gid": "0", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_root_ino_uid": "0", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_scrub_stats_review_period": "1", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server_dispatch_client_request_delay": "0", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_server_dispatch_killpoint_random": "0.000000", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-15T21:31:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_shutdown_check": "0", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_skip_ino": "0", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_snap_max_uid": "4294967294", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_snap_min_uid": "0", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_snap_rstat": "false", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_standby_replay_damaged": "false", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_symlink_recovery": "true", 2024-09-15T21:31:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_thrash_exports": "0", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_thrash_fragments": "0", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_tick_interval": "5.000000", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_valgrind_exit": "false", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_verify_backtrace": "1", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_verify_scatter": "false", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mds_wipe_sessions": "false", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mempool_debug": "false", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "memstore_device_bytes": "1073741824", 2024-09-15T21:31:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "memstore_page_set": "false", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "memstore_page_size": "65536", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_client_bytes": "134217728", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_client_messages": "512", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_disabled_modules": "", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_enable_op_tracker": "true", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_max_pg_creating": "1024", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_max_pg_num_change": "128", 2024-09-15T21:31:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_mds_bytes": "134217728", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_mds_messages": "128", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_mon_bytes": "134217728", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_mon_messages": "128", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_op_history_duration": "600", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_op_history_size": "20", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_op_log_threshold": "5", 2024-09-15T21:31:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_osd_bytes": "536870912", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_osd_messages": "8192", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_pool": "true", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_standby_modules": "true", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_stats_period": "5", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_stats_threshold": "5", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_tick_period": "2", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_allow_pool_delete": "true", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_allow_pool_size_one": "true", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_auth_validate_all_caps": "true", 2024-09-15T21:31:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_bytes": "104857600", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_directed_command_retry": "5", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_hunt_parallel": "3", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_log_interval": "1.000000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_ping_interval": "10.000000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-15T21:31:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_client_target_rank": "-1", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_level": "debug", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_to_file": "true", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_to_journald": "false", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-15T21:31:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_compact_on_bootstrap": "false", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_compact_on_start": "false", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_compact_on_trim": "true", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_cpu_threads": "4", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_crush_min_required_version": "hammer", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_daemon_bytes": "419430400", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_data_avail_crit": "5", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_data_avail_warn": "5", 2024-09-15T21:31:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_data_size_warn": "16106127360", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_dump_json": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_dump_transactions": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_extra_checks": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_no_require_reef": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_no_require_squid": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-15T21:31:41.586 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_down_mkfs_grace": "120", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_down_uptime_grace": "60", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_election_default_strategy": "1", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_election_timeout": "5.000000", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_elector_ping_divisor": "2", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_enable_op_tracker": "true", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_fake_pool_delete": "false", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_force_quorum_join": "false", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_globalid_prealloc": "10000", 2024-09-15T21:31:41.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_health_detail_to_clog": "false", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_health_log_update_period": "5", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_health_max_detail": "50", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_health_to_clog": "true", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_health_to_clog_interval": "600", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_host": "172.21.15.19,[v2:172.21.15.19:3301,v1:172.21.15.19:6790],172.21.15.192", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_host_override": "", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_initial_members": "", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-15T21:31:41.588 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_lease": "5.000000", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_log_full_interval": "50", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_log_max": "10000", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_log_max_summary": "50", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_log_epochs": "500", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_nvmeof_epochs": "500", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_osd": "10000", 2024-09-15T21:31:41.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_pg_per_osd": "10000", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_pool_pg_num": "65536", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mds_force_trim_to": "0", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mds_skip_sanity": "false", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_memory_autotune": "true", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_memory_target": "2147483648", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mgr_beacon_grace": "30", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mgr_digest_period": "5", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mgr_inactive_grace": "60", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-15T21:31:41.590 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_min_osdmap_epochs": "50", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_nvmeofgw_beacon_grace": "10", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_nvmeofgw_set_group_id_retry": "1000", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_op_complaint_time": "30", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_op_history_duration": "600", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_op_history_size": "20", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_op_history_slow_op_size": "20", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_op_log_threshold": "5", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-15T21:31:41.591 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_auto_mark_in": "false", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_cache_size": "500", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_down_out_interval": "600", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_force_trim_to": "0", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-15T21:31:41.592 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_min_down_reporters": "2", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-15T21:31:41.593 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_report_timeout": "900", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_pg_stuck_threshold": "60", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-15T21:31:41.594 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_probe_timeout": "2.000000", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_reweight_max_change": "0.050000", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_reweight_max_osds": "4", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-15T21:31:41.595 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_scrub_interval": "300", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_scrub_max_keys": "100", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_scrub_timeout": "300", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_session_timeout": "300", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_smart_report_timeout": "5", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_stat_smooth_intervals": "6", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_stretch_pool_min_size": "2", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_stretch_pool_size": "4", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_sync_debug": "false", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-15T21:31:41.596 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_sync_provider_kill_at": "0", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_sync_requester_kill_at": "0", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_sync_timeout": "60.000000", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_target_pg_per_osd": "100", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_tick_interval": "5", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_timecheck_interval": "300.000000", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_use_min_delay_socket": "false", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_older_version_delay": "604800", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-15T21:31:41.597 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_misplaced": "false", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_older_version": "true", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-15T21:31:41.598 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "monmap": "", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "motr_tracing_enabled": "false", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_op_threads": "3", 2024-09-15T21:31:41.599 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_cm": "false", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_device_name": "", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_dscp": "96", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_local_gid": "", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_port_num": "1", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-15T21:31:41.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_sl": "3", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_support_srq": "true", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_rdma_type": "ib", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_async_reap_threshold": "5", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_before_connect": "false", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_ipv4": "true", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_ipv6": "false", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_msgr1": "true", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_msgr2": "true", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_port_max": "7568", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_port_min": "6800", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-15T21:31:41.601 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_retry_count": "3", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_bind_retry_delay": "5", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_blackhole_client": "false", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_blackhole_mds": "false", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_blackhole_mgr": "false", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_blackhole_mon": "false", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_blackhole_osd": "false", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_client_mode": "crc secure", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_client_throttle_retry_time_interval": "5000", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_cluster_mode": "crc secure", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_cluster_type": "", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_compress_secure": "false", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_connection_idle_timeout": "900", 2024-09-15T21:31:41.602 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_connection_ready_timeout": "10", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_crc_data": "true", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_crc_header": "true", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_die_on_bad_msg": "false", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_die_on_bug": "true", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_die_on_old_message": "true", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_die_on_skipped_message": "false", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-15T21:31:41.603 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_coremask": "0xF", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_enable_tso": "true", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_hugepages": "", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_lro": "true", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_memory_channel": "4", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_pmd": "", 2024-09-15T21:31:41.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_port_id": "0", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_dump_on_send": "false", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_initial_backoff": "0.200000", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_inject_delay_max": "1.000000", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_inject_delay_type": "", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_inject_network_congestion": "0", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_inject_socket_failures": "5000", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_learn_addr_from_peer": "true", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_max_accept_failures": "4", 2024-09-15T21:31:41.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_max_backoff": "15.000000", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_mon_client_mode": "secure crc", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_mon_service_mode": "secure crc", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_osd_compress_min_size": "1024", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_osd_compress_mode": "none", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_pq_min_cost": "65536", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_public_type": "", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_service_mode": "crc secure", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_tcp_listen_backlog": "512", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_tcp_nodelay": "true", 2024-09-15T21:31:41.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_tcp_rcvbuf": "0", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_time_events_min_wait_interval": "1000", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "ms_type": "async", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "no_config_file": "false", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "nvmeof_mon_client_disconnect_panic": "100", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "nvmeof_mon_client_tick_period": "2", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter_completion_locks_per_session": "32", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter_inflight_ops": "1024", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-15T21:31:41.607 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter_tick_interval": "5.000000", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "objecter_timeout": "10.000000", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "objectstore_blackhole": "false", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "openssl_engine_opts": "", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_agent_delay_time": "5.000000", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_agent_hist_halflife": "1000", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_agent_max_low_ops": "2", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_agent_max_ops": "4", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_agent_slop": "0.020000", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-15T21:31:41.608 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_asio_thread_count": "2", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_async_recovery_min_cost": "100", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_backfill_scan_max": "512", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_backfill_scan_min": "64", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_backoff_on_degraded": "false", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_backoff_on_peering": "false", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_backoff_on_unfound": "true", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_beacon_report_interval": "300", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bench_duration": "30", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-15T21:31:41.609 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bench_max_block_size": "67108864", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_blkin_trace_all": "false", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_check_for_log_corruption": "false", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-15T21:31:41.610 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_class_update_on_start": "true", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_client_message_cap": "256", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_client_message_size_cap": "524288000", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_client_op_priority": "63", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_client_watch_timeout": "30", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_command_max_records": "256", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_compact_on_start": "false", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_crush_update_on_start": "true", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_crush_update_weight_set": "true", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-15T21:31:41.611 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_feed_pullee": "-1", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_misdirected_ops": "false", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_no_acting_change": "false", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_no_purge_strays": "false", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_op_order": "false", 2024-09-15T21:31:41.612 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_shutdown": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_trim_objects": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_verify_snaps": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-15T21:31:41.613 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_deep_scrub_keys": "1024", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_deep_scrub_stride": "524288", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_default_notify_timeout": "30", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_delete_sleep": "0.000000", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_discard_disconnected_ops": "true", 2024-09-15T21:31:41.614 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_ec_partial_reads": "true", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_enable_op_tracker": "true", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_fast_info": "true", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_fast_shutdown": "true", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-15T21:31:41.615 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_function_tracing": "false", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_heartbeat_grace": "20", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_heartbeat_interval": "6", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_heartbeat_min_peers": "10", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_heartbeat_min_size": "2000", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_heartbeat_stale": "600", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_hit_set_max_size": "100000", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_hit_set_min_size": "1000", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-15T21:31:41.616 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_journal_size": "5120", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_kill_backfill_at": "0", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_map_cache_size": "50", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_map_dedup": "true", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_map_message_max": "40", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_map_share_max_epochs": "40", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_attr_name_len": "100", 2024-09-15T21:31:41.617 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_attr_size": "0", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_backfills": "1", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_markdown_count": "5", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_markdown_period": "600", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_object_name_len": "2048", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_object_namespace_len": "256", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_object_size": "134217728", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_pg_blocked_by": "16", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_pg_log_entries": "10000", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-15T21:31:41.618 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_pgls": "1024", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_push_cost": "8388608", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_push_objects": "10", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_scrubs": "3", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_trimming_pgs": "2", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_write_op_reply_len": "64", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_max_write_size": "90", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-15T21:31:41.619 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_profile": "balanced", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-15T21:31:41.620 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_memory_base": "805306368", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_memory_cache_min": "134217728", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_memory_target": "4294967296", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_memory_target_autotune": "false", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_min_pg_log_entries": "250", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_min_recovery_priority": "0", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-15T21:31:41.621 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mon_report_interval": "5", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_num_cache_shards": "32", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_num_op_tracker_shard": "32", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_numa_auto_affinity": "true", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_numa_node": "-1", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_numa_prefer_iface": "true", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_objecter_finishers": "1", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_objectstore": "bluestore", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_objectstore_fuse": "false", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_objectstore_ideal_list_max": "64", 2024-09-15T21:31:41.622 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_objectstore_tracing": "false", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_complaint_time": "30.000000", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_history_duration": "600", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_history_size": "20", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_history_slow_op_size": "20", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_log_threshold": "5", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_num_shards": "0", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_num_shards_hdd": "5", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_num_shards_ssd": "8", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-15T21:31:41.623 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_pq_min_cost": "65536", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_queue_cut_off": "high", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_op_thread_timeout": "15", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_open_classes_on_start": "true", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_os_flags": "0", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_peering_op_priority": "255", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_delete_cost": "1048576", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_delete_priority": "5", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-15T21:31:41.624 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_log_trim_max": "10000", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_log_trim_min": "100", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-15T21:31:41.625 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_crimson": "false", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_flags": "0", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_min_size": "0", 2024-09-15T21:31:41.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_pg_num": "32", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_pgp_num": "0", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_read_ratio": "70", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_size": "2", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_default_type": "replicated", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_push_per_object_cost": "1000", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recover_clone_overlap": "true", 2024-09-15T21:31:41.627 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_cost": "20971520", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_max_active": "0", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_max_single_start": "1", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_op_priority": "3", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_priority": "5", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-15T21:31:41.628 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_sleep": "0.000000", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_repair_during_recovery": "false", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_requested_scrub_priority": "120", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_auto_repair": "false", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_begin_hour": "0", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_begin_week_day": "0", 2024-09-15T21:31:41.629 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_chunk_max": "15", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_chunk_min": "5", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_cost": "52428800", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_during_recovery": "false", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_end_hour": "0", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_end_week_day": "0", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_event_cost": "4096", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_invalid_stats": "true", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-15T21:31:41.630 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_max_preemptions": "5", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_priority": "5", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_retry_after_noscrub": "60", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_retry_delay": "30", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_retry_new_interval": "10", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_retry_pg_state": "60", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_retry_trimming": "10", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_scrub_sleep": "0.000000", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-15T21:31:41.631 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_skip_check_past_interval_bounds": "false", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_skip_data_digest": "false", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_smart_report_timeout": "5", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_snap_trim_cost": "1048576", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_snap_trim_priority": "5", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_target_transaction_size": "30", 2024-09-15T21:31:41.632 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_tracing": "false", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_use_stale_snap": "false", 2024-09-15T21:31:41.633 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "osdc_blkin_trace_all": "false", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_kill_at": "0", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_max_join_drift": "10", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_min": "500", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_min_wait": "0.050000", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_propose_interval": "1.000000", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_service_trim_max": "500", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_service_trim_min": "10", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_stash_full_interval": "25", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_trim_max": "500", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "paxos_trim_min": "250", 2024-09-15T21:31:41.634 INFO:teuthology.orchestra.run.smithi019.stdout: "perf": "true", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "-", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrv": "", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "public_bind_addr": "-", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "public_network": "", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "public_network_interface": "", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "qat_compressor_busy_polling": "false", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "qat_compressor_enabled": "false", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "qat_compressor_session_max_number": "256", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "rados_mon_op_timeout": "0", 2024-09-15T21:31:41.635 INFO:teuthology.orchestra.run.smithi019.stdout: "rados_osd_op_timeout": "0", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rados_tracing": "false", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_atime_update_interval": "60", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_balance_parent_reads": "false", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_balance_snap_reads": "false", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_blkin_trace_all": "false", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache": "true", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-15T21:31:41.636 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache_policy": "writearound", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache_size": "33554432", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_clone_copy_on_read": "false", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_compression_hint": "none", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_concurrent_management_ops": "10", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_clone_format": "auto", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_data_pool": "", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_features": "61", 2024-09-15T21:31:41.637 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_format": "2", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_map_options": "", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_order": "22", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_pool": "rbd", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_stripe_count": "0", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_default_stripe_unit": "0", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_enable_alloc_hint": "true", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-15T21:31:41.638 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_io_scheduler": "simple", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_order": "24", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_pool": "", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_journal_splay_width": "4", 2024-09-15T21:31:41.639 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_localize_parent_reads": "false", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_localize_snap_reads": "false", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-15T21:31:41.640 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-15T21:31:41.641 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_mtime_update_interval": "60", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_non_blocking_aio": "true", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_op_thread_timeout": "60", 2024-09-15T21:31:41.642 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_op_threads": "1", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_parent_cache_enabled": "false", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_plugins": "", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_bps_burst": "0", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_bps_limit": "0", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_exclude_ops": "0", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_iops_burst": "0", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_iops_limit": "0", 2024-09-15T21:31:41.643 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-15T21:31:41.644 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_read_from_replica_policy": "default", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_skip_partial_discard": "true", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_tracing": "false", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_validate_names": "true", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd_validate_pool": "true", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "restapi_base_url": "", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "restapi_log_level": "", 2024-09-15T21:31:41.645 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_acl_grants_max_num": "100", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_admin_entry": "admin", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_asio_assert_yielding": "false", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_backend_store": "rados", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_barbican_url": "", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_beast_enable_async": "true", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_counters_cache": "false", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-15T21:31:41.646 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_cache_enabled": "true", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_cache_expiry_interval": "900", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_cache_lru_size": "25000", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_config_store": "rados", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_content_length_compat": "false", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_copy_obj_progress": "true", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-15T21:31:41.647 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_cors_rules_max_num": "100", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_cross_domain_policy": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_kmip_addr": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_kmip_password": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_kmip_username": "", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_require_ssl": "true", 2024-09-15T21:31:41.648 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-15T21:31:41.649 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_addr": "", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_auth": "token", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_namespace": "", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_prefix": "", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_token_file": "", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_curl_buffersize": "524288", 2024-09-15T21:31:41.650 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_curl_low_speed_time": "300", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d4n_address": "127.0.0.1:6379", 2024-09-15T21:31:41.651 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d4n_l1_datacache_address": "127.0.0.1:6379", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d4n_l1_datacache_persistent_path": "/tmp/rgw_d4n_datacache/", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d4n_l1_datacache_size": "1073741824", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d4n_l1_evict_cache_on_start": "true", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d4n_l1_fadvise": "4", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d4n_libaio_aio_num": "64", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_d4n_libaio_aio_threads": "20", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_data_log_changes_size": "1000", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_data_log_num_shards": "128", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_data_log_window": "30", 2024-09-15T21:31:41.652 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-15T21:31:41.653 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_disable_s3select": "false", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-15T21:31:41.654 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dns_name": "", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dns_s3website_name": "", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dynamic_resharding": "true", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dynamic_resharding_may_reduce": "true", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_dynamic_resharding_reduction_wait": "120", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enable_gc_threads": "true", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enable_lc_threads": "true", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enable_mdsearch": "true", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enable_ops_log": "false", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enable_quota_threads": "true", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enable_static_website": "false", 2024-09-15T21:31:41.655 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enable_usage_log": "false", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_enforce_swift_acls": "true", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_exit_timeout_secs": "120", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_expose_bucket": "false", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_extended_http_attrs": "", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_filter": "none", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_frontends": "beast port=7480", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_max_deferred": "50", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_max_objs": "32", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-15T21:31:41.656 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_gc_processor_period": "3600", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_init_timeout": "300", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_kafka_connection_idle": "300", 2024-09-15T21:31:41.657 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_admin_domain": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_admin_password": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_admin_password_path": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_admin_project": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_admin_tenant": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_admin_token": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_admin_token_path": "", 2024-09-15T21:31:41.658 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_admin_user": "", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_barbican_domain": "", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_barbican_password": "", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_barbican_project": "", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_barbican_user": "", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_url": "", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-15T21:31:41.659 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lc_debug_interval": "-1", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lc_lock_max_time": "90", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lc_max_objs": "32", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lc_max_rules": "1000", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lc_max_worker": "3", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lc_thread_delay": "0", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ldap_dnattr": "uid", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ldap_searchfilter": "", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ldap_uri": "", 2024-09-15T21:31:41.660 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lfuda_sync_frequency": "60", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_log_http_headers": "", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_log_object_name_utc": "false", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_attr_name_len": "0", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_attr_size": "0", 2024-09-15T21:31:41.661 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_chunk_size": "4194304", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_listing_results": "5000", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_notify_retries": "10", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_put_param_size": "1048576", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_put_size": "5368709120", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_max_slo_entries": "1000", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_md_log_max_shards": "64", 2024-09-15T21:31:41.662 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_mp_lock_max_time": "600", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-15T21:31:41.663 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_max_gc": "300", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_num_async_rados_threads": "32", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_num_control_oids": "8", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_numa_node": "-1", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-15T21:31:41.664 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_objexp_chunk_size": "100", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_objexp_gc_interval": "600", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_op_thread_timeout": "600", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_op_tracing": "false", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_opa_token": "", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_opa_url": "", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_opa_verify_ssl": "true", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-15T21:31:41.665 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ops_log_rados": "false", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_ops_log_socket_path": "", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_period_push_interval": "2.000000", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_posix_cache_lanes": "3", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-15T21:31:41.666 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_posix_cache_partitions": "3", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_print_continue": "true", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_rados_tracing": "false", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_realm": "", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_realm_id": "", 2024-09-15T21:31:41.667 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_region": "", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_request_uri": "", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshard_batch_size": "64", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshard_debug_interval": "-1", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshard_max_aio": "128", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshard_num_logs": "16", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshard_progress_judge_interval": "120", 2024-09-15T21:31:41.668 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshard_progress_judge_ratio": "0.500000", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshard_thread_interval": "600", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_reshardlog_threshold": "30000", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_resolve_cname": "false", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_run_sync_thread": "true", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-15T21:31:41.669 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_scheduler_type": "throttler", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_script_uri": "", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_service_provider_name": "", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sts_client_id": "", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sts_client_secret": "", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sts_entry": "sts", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sts_key": "sts", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sts_min_session_duration": "900", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sts_token_introspection_url": "", 2024-09-15T21:31:41.670 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_account_in_url": "false", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_auth_entry": "auth", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_auth_url": "", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_custom_header": "", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_need_stats": "true", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_tenant_name": "", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_token_expiration": "86400", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_url": "", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_url_prefix": "swift", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-15T21:31:41.671 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_lease_period": "120", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_thread_pool_size": "512", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-15T21:31:41.672 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_torrent_comment": "", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_torrent_createby": "", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_torrent_encoding": "", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_torrent_flag": "false", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_torrent_origin": "", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_torrent_tracker": "", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_trust_forwarded_https": "false", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-15T21:31:41.673 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_usage_max_shards": "32", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_usage_max_user_shards": "1", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_use_opa_authz": "false", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_counters_cache": "false", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_max_buckets": "1000", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_policies_max_num": "100", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-15T21:31:41.674 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_user_unique_email": "true", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_verify_ssl": "true", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_zone": "", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_zone_id": "", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_zonegroup": "", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_zonegroup_id": "", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgwlc_auto_session_clear": "true", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_block_size": "4096", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-15T21:31:41.675 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cache_size": "536870912", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-15T21:31:41.676 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_index_type": "binary_search", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_partition_filters": "false", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_perf": "false", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "run_dir": "/var/run/ceph", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_block_create": "true", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_cache_lru_size": "67108864", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-15T21:31:41.677 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_default_max_object_size": "16777216", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_device_size": "53687091200", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_full_integrity_check": "false", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_journal_batch_capacity": "16", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_main_device_type": "SSD", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-15T21:31:41.678 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "seastore_segment_size": "67108864", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "set_keepcaps": "false", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "setgroup": "", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "setuser": "", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "setuser_match_path": "", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "thp": "false", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "threadpool_default_timeout": "60", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "throttler_perf_counter": "true", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tmp_dir": "/tmp", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "tmp_file_template": "/tmp/ceph-mon.a.XXXXXX", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "uadk_compressor_enabled": "false", 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout: "uadk_wd_sync_ctx_num": "2" 2024-09-15T21:31:41.679 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:41.681 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-15T21:31:41.797 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a: {} 2024-09-15T21:31:41.798 INFO:teuthology.orchestra.run.smithi019.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:31:41.807 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b: {} 2024-09-15T21:31:41.807 INFO:teuthology.orchestra.run.smithi019.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:31:41.815 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c: {} 2024-09-15T21:31:41.816 INFO:teuthology.orchestra.run.smithi019.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:31:41.826 DEBUG:teuthology.orchestra.run.smithi192:> type ceph-objectstore-tool 2024-09-15T21:31:41.844 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T21:31:41.845 DEBUG:teuthology.orchestra.run.smithi019:> type ceph-objectstore-tool 2024-09-15T21:31:41.859 INFO:teuthology.orchestra.run.smithi019.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-15T21:31:41.860 INFO:teuthology.run_tasks:Running task mon_recovery... 2024-09-15T21:31:41.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:31:41.874 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-15T21:31:41.874 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:31:41.874 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:31:41.874 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T21:31:41.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T21:31:41.875 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-15T21:31:41.875 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-15T21:31:41.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:31:41.876 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-15T21:31:41.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:31:41.876 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-15T21:31:41.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:31:41.902 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:41.899+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:41.977 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:41.975+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:42.020 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:31:42.021 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:42.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:31:42.049 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:31:42.049 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:31:42.073 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:31:42.073 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:31:42.077 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:42.075+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:42.091 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:31:42.092 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:31:42.108 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:31:42.109 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:31:42.132 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:31:42.132 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:31:42.154 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:31:42.154 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:31:42.178 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:42.176+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:42.184 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:31:42.185 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:31:42.193 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:42.193 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:42.202 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:31:42.219 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:31:42.220 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:31:42.269 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:42.270 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":23,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:31:42.112969+0000","last_up_change":"2024-09-15T21:31:25.414443+0000","last_in_change":"2024-09-15T21:31:14.500069+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:31:22.622713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:31:30.313202+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6825","nonce":368705522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6827","nonce":368705522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6831","nonce":368705522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6829","nonce":368705522}]},"public_addr":"172.21.15.19:6825/368705522","cluster_addr":"172.21.15.19:6827/368705522","heartbeat_back_addr":"172.21.15.19:6831/368705522","heartbeat_front_addr":"172.21.15.19:6829/368705522","state":["exists","up"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6817","nonce":3888640163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6819","nonce":3888640163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6823","nonce":3888640163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6821","nonce":3888640163}]},"public_addr":"172.21.15.19:6817/3888640163","cluster_addr":"172.21.15.19:6819/3888640163","heartbeat_back_addr":"172.21.15.19:6823/3888640163","heartbeat_front_addr":"172.21.15.19:6821/3888640163","state":["exists","up"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6809","nonce":1831747042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6811","nonce":1831747042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6815","nonce":1831747042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6813","nonce":1831747042}]},"public_addr":"172.21.15.19:6809/1831747042","cluster_addr":"172.21.15.19:6811/1831747042","heartbeat_back_addr":"172.21.15.19:6815/1831747042","heartbeat_front_addr":"172.21.15.19:6813/1831747042","state":["exists","up"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6801","nonce":3658047863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6803","nonce":3658047863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6807","nonce":3658047863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6805","nonce":3658047863}]},"public_addr":"172.21.15.19:6801/3658047863","cluster_addr":"172.21.15.19:6803/3658047863","heartbeat_back_addr":"172.21.15.19:6807/3658047863","heartbeat_front_addr":"172.21.15.19:6805/3658047863","state":["exists","up"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6817","nonce":1988554297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6819","nonce":1988554297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6823","nonce":1988554297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6821","nonce":1988554297}]},"public_addr":"172.21.15.192:6817/1988554297","cluster_addr":"172.21.15.192:6819/1988554297","heartbeat_back_addr":"172.21.15.192:6823/1988554297","heartbeat_front_addr":"172.21.15.192:6821/1988554297","state":["exists","up"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6825","nonce":1071217311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6827","nonce":1071217311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6831","nonce":1071217311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6829","nonce":1071217311}]},"public_addr":"172.21.15.192:6825/1071217311","cluster_addr":"172.21.15.192:6827/1071217311","heartbeat_back_addr":"172.21.15.192:6831/1071217311","heartbeat_front_addr":"172.21.15.192:6829/1071217311","state":["exists","up"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6801","nonce":1069316293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6803","nonce":1069316293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6807","nonce":1069316293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6805","nonce":1069316293}]},"public_addr":"172.21.15.192:6801/1069316293","cluster_addr":"172.21.15.192:6803/1069316293","heartbeat_back_addr":"172.21.15.192:6807/1069316293","heartbeat_front_addr":"172.21.15.192:6805/1069316293","state":["exists","up"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6809","nonce":3119270077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6811","nonce":3119270077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6815","nonce":3119270077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6813","nonce":3119270077}]},"public_addr":"172.21.15.192:6809/3119270077","cluster_addr":"172.21.15.192:6811/3119270077","heartbeat_back_addr":"172.21.15.192:6815/3119270077","heartbeat_front_addr":"172.21.15.192:6813/3119270077","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.795411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.139237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.562583+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.363254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.344635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.323050+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.381780+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.049941+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:31:42.279 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:42.277+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:42.283 INFO:tasks.mon_recovery.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T21:31:22.622713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:31:30.313202+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T21:31:42.283 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-15T21:31:42.329 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:42.329 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:42.333 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:42.333 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:42.333 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 23, 2024-09-15T21:31:42.333 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:31:42.333 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2024-09-15T21:31:12.329884+0000", 2024-09-15T21:31:42.333 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2024-09-15T21:31:42.112969+0000", 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "last_up_change": "2024-09-15T21:31:25.414443+0000", 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "last_in_change": "2024-09-15T21:31:14.500069+0000", 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_num": 5799936, 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_set": [ 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "pglog_hardlimit", 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "purged_snapdirs", 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_deletes", 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "sortbitwise" 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_version": 5, 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:31:42.334 INFO:teuthology.orchestra.run.smithi019.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_snapshot": "", 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_max": 2, 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd": 8, 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "require_min_compat_client": "luminous", 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "min_compat_client": "jewel", 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "require_osd_release": "squid", 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_crimson": false, 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "pools": [ 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 1, 2024-09-15T21:31:42.335 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": ".mgr", 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:22.622713+0000", 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool", 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:31:42.336 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 1, 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 1, 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 1, 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 1, 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 1, 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:31:42.337 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "18", 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:31:42.338 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 0, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 0, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:31:42.339 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:31:42.340 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "options": { 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_max": 32, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_min": 1 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:31:42.341 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": {} 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 8, 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 8, 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 0.25, 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 2, 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 2, 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 1, 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 1, 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:31:42.342 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 2, 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": "rbd", 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:30.313202+0000", 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 8193, 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:31:42.343 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 8, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 8, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 8, 2024-09-15T21:31:42.344 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 8, 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 8, 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "22", 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:31:42.345 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 2, 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 22, 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:31:42.346 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:31:42.347 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:31:42.348 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "options": {}, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd": {} 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 1.9900000095367432, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 1.9900000095367432, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 0.87999999523162842, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 1.75, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 1.75, 2024-09-15T21:31:42.349 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 1, 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 1, 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 1 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "osds": [ 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "f7eb9035-e8c2-4d90-a198-2bed78d5a3b5", 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:31:42.350 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6824", 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6825", 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6826", 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:31:42.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6827", 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6830", 2024-09-15T21:31:42.353 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6831", 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.354 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6828", 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6829", 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6825/368705522", 2024-09-15T21:31:42.355 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6827/368705522", 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6831/368705522", 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6829/368705522", 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a", 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:31:42.356 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 19, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:42.357 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.362 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6816", 2024-09-15T21:31:42.362 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:31:42.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.362 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6817", 2024-09-15T21:31:42.363 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:31:42.363 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.363 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.363 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:31:42.363 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6818", 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6819", 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.364 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6822", 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6823", 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6820", 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6821", 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.366 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6817/3888640163", 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6819/3888640163", 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6823/3888640163", 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6821/3888640163", 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:31:42.367 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "180bae67-0f64-45a0-9a58-f4b8ff2b90a5", 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 19, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:42.368 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6808", 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6809", 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:31:42.369 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6810", 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6811", 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.370 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6814", 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.371 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6815", 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6812", 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6813", 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6809/1831747042", 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6811/1831747042", 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6815/1831747042", 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6813/1831747042", 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:31:42.373 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d05cec44-e228-4297-98c8-53e2ad91b37c", 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:31:42.374 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-09-15T21:31:42.375 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6802", 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.376 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6803", 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6806", 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6807", 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6804", 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.378 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6805", 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6801/3658047863", 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6803/3658047863", 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6807/3658047863", 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6805/3658047863", 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:31:42.379 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "5d7097a2-8165-4885-a356-f8e67851be5d", 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:31:42.380 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6816", 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6817", 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6818", 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:31:42.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6819", 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6822", 2024-09-15T21:31:42.383 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6823", 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.384 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6820", 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6821", 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6817/1988554297", 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6819/1988554297", 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1988554297", 2024-09-15T21:31:42.385 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1988554297", 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f", 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:31:42.386 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 14, 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 19, 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6824", 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.387 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6825", 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6826", 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:31:42.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6827", 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6830", 2024-09-15T21:31:42.389 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6831", 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6828", 2024-09-15T21:31:42.390 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6829", 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6825/1071217311", 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6827/1071217311", 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6831/1071217311", 2024-09-15T21:31:42.391 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6829/1071217311", 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612", 2024-09-15T21:31:42.392 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 19, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6800", 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6801", 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.394 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6802", 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6803", 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:31:42.395 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6806", 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6807", 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6804", 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6805", 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6801/1069316293", 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6803/1069316293", 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6807/1069316293", 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6805/1069316293", 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:31:42.398 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:31:42.399 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:31:42.399 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.399 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:31:42.399 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "fb77f072-c3ae-4914-81bd-473d30016320", 2024-09-15T21:31:42.399 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:31:42.400 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:31:42.400 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:31:42.400 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:31:42.400 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:31:42.400 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:31:42.400 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:31:42.400 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 19, 2024-09-15T21:31:42.400 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6808", 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6809", 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:31:42.401 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6810", 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6811", 2024-09-15T21:31:42.402 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6814", 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.404 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.404 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6815", 2024-09-15T21:31:42.404 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:31:42.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.404 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6812", 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:42.405 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6813", 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6809/3119270077", 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6811/3119270077", 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6815/3119270077", 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6813/3119270077", 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.406 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_xinfo": [ 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.795411+0000", 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.407 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.139237+0000", 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:31:42.408 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.562583+0000", 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:31:42.409 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.363254+0000", 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.344635+0000", 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.323050+0000", 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:31:42.411 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.381780+0000", 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:31:42.412 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.049941+0000", 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap": [], 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_items": [], 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_primaries": [], 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_temp": [], 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_temp": [], 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklist": {}, 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "range_blocklist": {}, 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profiles": { 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "default": { 2024-09-15T21:31:42.413 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-failure-domain": "osd", 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "k": "2", 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "m": "1", 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "plugin": "jerasure", 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "technique": "reed_sol_van" 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps_queue": [], 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "new_removed_snaps": [], 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "new_purged_snaps": [ 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 2, 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: "snaps": [ 2024-09-15T21:31:42.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "begin": 2, 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "length": 1 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_node_flags": {}, 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "device_class_flags": {}, 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": { 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_enabled": false, 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_bucket_count": 0, 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:31:42.415 INFO:teuthology.orchestra.run.smithi019.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:31:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_bucket": 0 2024-09-15T21:31:42.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:42.416 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:42.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:31:42.418 INFO:tasks.thrashosds.thrasher:j is {'epoch': 23, 'fsid': '13019d66-d4b6-4e2a-8322-38bcb4210bcf', 'created': '2024-09-15T21:31:12.329884+0000', 'modified': '2024-09-15T21:31:42.112969+0000', 'last_up_change': '2024-09-15T21:31:25.414443+0000', 'last_in_change': '2024-09-15T21:31:14.500069+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T21:31:22.622713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:31:30.313202+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'f7eb9035-e8c2-4d90-a198-2bed78d5a3b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6824', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6825', 'nonce': 368705522}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6826', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6827', 'nonce': 368705522}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6830', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6831', 'nonce': 368705522}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6828', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6829', 'nonce': 368705522}]}, 'public_addr': '172.21.15.19:6825/368705522', 'cluster_addr': '172.21.15.19:6827/368705522', 'heartbeat_back_addr': '172.21.15.19:6831/368705522', 'heartbeat_front_addr': '172.21.15.19:6829/368705522', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd2bcecd4-0bb0-4b38-b395-8b6dbb48f53a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6816', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6817', 'nonce': 3888640163}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6818', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6819', 'nonce': 3888640163}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6822', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6823', 'nonce': 3888640163}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6820', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6821', 'nonce': 3888640163}]}, 'public_addr': '172.21.15.19:6817/3888640163', 'cluster_addr': '172.21.15.19:6819/3888640163', 'heartbeat_back_addr': '172.21.15.19:6823/3888640163', 'heartbeat_front_addr': '172.21.15.19:6821/3888640163', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '180bae67-0f64-45a0-9a58-f4b8ff2b90a5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6808', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6809', 'nonce': 1831747042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6810', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6811', 'nonce': 1831747042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6814', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6815', 'nonce': 1831747042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6812', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6813', 'nonce': 1831747042}]}, 'public_addr': '172.21.15.19:6809/1831747042', 'cluster_addr': '172.21.15.19:6811/1831747042', 'heartbeat_back_addr': '172.21.15.19:6815/1831747042', 'heartbeat_front_addr': '172.21.15.19:6813/1831747042', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd05cec44-e228-4297-98c8-53e2ad91b37c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6800', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6801', 'nonce': 3658047863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6802', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6803', 'nonce': 3658047863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6806', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6807', 'nonce': 3658047863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6804', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6805', 'nonce': 3658047863}]}, 'public_addr': '172.21.15.19:6801/3658047863', 'cluster_addr': '172.21.15.19:6803/3658047863', 'heartbeat_back_addr': '172.21.15.19:6807/3658047863', 'heartbeat_front_addr': '172.21.15.19:6805/3658047863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '5d7097a2-8165-4885-a356-f8e67851be5d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1988554297}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1988554297}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1988554297}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1988554297}]}, 'public_addr': '172.21.15.192:6817/1988554297', 'cluster_addr': '172.21.15.192:6819/1988554297', 'heartbeat_back_addr': '172.21.15.192:6823/1988554297', 'heartbeat_front_addr': '172.21.15.192:6821/1988554297', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 1071217311}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 1071217311}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 1071217311}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 1071217311}]}, 'public_addr': '172.21.15.192:6825/1071217311', 'cluster_addr': '172.21.15.192:6827/1071217311', 'heartbeat_back_addr': '172.21.15.192:6831/1071217311', 'heartbeat_front_addr': '172.21.15.192:6829/1071217311', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 1069316293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 1069316293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 1069316293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 1069316293}]}, 'public_addr': '172.21.15.192:6801/1069316293', 'cluster_addr': '172.21.15.192:6803/1069316293', 'heartbeat_back_addr': '172.21.15.192:6807/1069316293', 'heartbeat_front_addr': '172.21.15.192:6805/1069316293', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'fb77f072-c3ae-4914-81bd-473d30016320', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 3119270077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 3119270077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 3119270077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 3119270077}]}, 'public_addr': '172.21.15.192:6809/3119270077', 'cluster_addr': '172.21.15.192:6811/3119270077', 'heartbeat_back_addr': '172.21.15.192:6815/3119270077', 'heartbeat_front_addr': '172.21.15.192:6813/3119270077', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.795411+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.139237+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.562583+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.363254+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.344635+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.323050+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.381780+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.049941+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [{'pool': 2, 'snaps': [{'begin': 2, 'length': 1}]}], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T21:31:42.419 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:42.419 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:42.415+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:42.517 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:42.515+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:42.571 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:42.571 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:42.580 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:31:42.619 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:42.617+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:42.634 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 1 2024-09-15T21:31:42.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-15T21:31:42.721 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:42.718+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:42.734 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:42.734 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:42.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:31:42.769 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:31:42.770 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:31:42.783 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-15T21:31:42.610432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218080,"kb_used_data":1808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754756640,"statfs":{"total":773094113280,"available":772870799360,"internally_reserved":0,"allocated":1851392,"data_stored":1665462,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.152341"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602188+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602188+0000","last_peered":"2024-09-15T21:31:33.602188+0000","last_clean":"2024-09-15T21:31:33.602188+0000","last_became_active":"2024-09-15T21:31:31.455107+0000","last_became_peered":"2024-09-15T21:31:31.455107+0000","last_unstale":"2024-09-15T21:31:33.602188+0000","last_undegraded":"2024-09-15T21:31:33.602188+0000","last_fullsized":"2024-09-15T21:31:33.602188+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:37:27.496347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465416+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:31:33.465416+0000","last_peered":"2024-09-15T21:31:33.465416+0000","last_clean":"2024-09-15T21:31:33.465416+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:31:33.465416+0000","last_undegraded":"2024-09-15T21:31:33.465416+0000","last_fullsized":"2024-09-15T21:31:33.465416+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602227+0000","last_change":"2024-09-15T21:31:33.602470+0000","last_active":"2024-09-15T21:31:33.602227+0000","last_peered":"2024-09-15T21:31:33.602227+0000","last_clean":"2024-09-15T21:31:33.602227+0000","last_became_active":"2024-09-15T21:31:31.458361+0000","last_became_peered":"2024-09-15T21:31:31.458361+0000","last_unstale":"2024-09-15T21:31:33.602227+0000","last_undegraded":"2024-09-15T21:31:33.602227+0000","last_fullsized":"2024-09-15T21:31:33.602227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:31:47.228609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000522709,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.465467+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:31:33.465467+0000","last_peered":"2024-09-15T21:31:33.465467+0000","last_clean":"2024-09-15T21:31:33.465467+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:31:33.465467+0000","last_undegraded":"2024-09-15T21:31:33.465467+0000","last_fullsized":"2024-09-15T21:31:33.465467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"22'2","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.464681+0000","last_change":"2024-09-15T21:31:33.463884+0000","last_active":"2024-09-15T21:31:33.464681+0000","last_peered":"2024-09-15T21:31:33.464681+0000","last_clean":"2024-09-15T21:31:33.464681+0000","last_became_active":"2024-09-15T21:31:31.458375+0000","last_became_peered":"2024-09-15T21:31:31.458375+0000","last_unstale":"2024-09-15T21:31:33.464681+0000","last_undegraded":"2024-09-15T21:31:33.464681+0000","last_fullsized":"2024-09-15T21:31:33.464681+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:55:21.745019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001154895,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.459144+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:31:33.459144+0000","last_peered":"2024-09-15T21:31:33.459144+0000","last_clean":"2024-09-15T21:31:33.459144+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:31:33.459144+0000","last_undegraded":"2024-09-15T21:31:33.459144+0000","last_fullsized":"2024-09-15T21:31:33.459144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.602178+0000","last_change":"2024-09-15T21:31:33.602420+0000","last_active":"2024-09-15T21:31:33.602178+0000","last_peered":"2024-09-15T21:31:33.602178+0000","last_clean":"2024-09-15T21:31:33.602178+0000","last_became_active":"2024-09-15T21:31:31.458374+0000","last_became_peered":"2024-09-15T21:31:31.458374+0000","last_unstale":"2024-09-15T21:31:33.602178+0000","last_undegraded":"2024-09-15T21:31:33.602178+0000","last_fullsized":"2024-09-15T21:31:33.602178+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:45.856320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049767300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"20'1","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.463323+0000","last_change":"2024-09-15T21:31:33.463323+0000","last_active":"2024-09-15T21:31:33.463323+0000","last_peered":"2024-09-15T21:31:33.463323+0000","last_clean":"2024-09-15T21:31:33.463323+0000","last_became_active":"2024-09-15T21:31:31.455121+0000","last_became_peered":"2024-09-15T21:31:31.455121+0000","last_unstale":"2024-09-15T21:31:33.463323+0000","last_undegraded":"2024-09-15T21:31:33.463323+0000","last_fullsized":"2024-09-15T21:31:33.463323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:01.950055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055004999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-15T21:31:33.601856+0000","last_change":"2024-09-15T21:31:26.424937+0000","last_active":"2024-09-15T21:31:33.601856+0000","last_peered":"2024-09-15T21:31:33.601856+0000","last_clean":"2024-09-15T21:31:33.601856+0000","last_became_active":"2024-09-15T21:31:26.424008+0000","last_became_peered":"2024-09-15T21:31:26.424008+0000","last_unstale":"2024-09-15T21:31:33.601856+0000","last_undegraded":"2024-09-15T21:31:33.601856+0000","last_fullsized":"2024-09-15T21:31:33.601856+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:05.634396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":1057856,"data_compressed":11036,"data_compressed_allocated":65536,"data_compressed_original":1024000,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476743,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476744,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476744,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":270336,"data_stored":604874,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":221184,"data_stored":75965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":217088,"data_stored":75946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:42.784 INFO:tasks.thrashosds.thrasher:Setting 2.2 to [6, 5] 2024-09-15T21:31:42.784 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.2', '6', '5'] 2024-09-15T21:31:42.784 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.2 6 5 2024-09-15T21:31:42.821 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:42.819+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:42.852 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:42.852 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:42.861 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:31:42.946 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:42.944+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:43.002 INFO:teuthology.orchestra.run.smithi019.stdout:pg_num: 8 2024-09-15T21:31:43.015 INFO:tasks.mon_recovery:mon ids = ['a', 'c', 'b'] 2024-09-15T21:31:43.015 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-09-15T21:31:43.015 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:43.023 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:43.021+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:43.059 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:43.059 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:43.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:31:43.121 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:31:43.124 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:43.122+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:43.224 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:43.223+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:43.225 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:43.225 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:43.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:31:43.325 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:43.323+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:43.341 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:43.341 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:43.350 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:31:43.426 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:43.425+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:43.432 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":31,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:43.446 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-09-15T21:31:43.446 INFO:tasks.mon_recovery:verifying all monitors are in the quorum 2024-09-15T21:31:43.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-15T21:31:43.459 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:43.459 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:43.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:31:43.527 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:43.526+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "a", 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 0, 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "leader", 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: "election_epoch": 8, 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 36465, 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum": [ 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: 0, 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: 1, 2024-09-15T21:31:43.573 INFO:teuthology.orchestra.run.smithi019.stdout: 2 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_age": 31, 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "features": { 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "required_mon": [ 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.574 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_mon": [ 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.575 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "outside_quorum": [], 2024-09-15T21:31:43.576 INFO:teuthology.orchestra.run.smithi019.stdout: "extra_probe_peers": [], 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "sync_provider": [], 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "monmap": { 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 1, 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2024-09-15T21:30:32.924046Z", 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2024-09-15T21:30:32.924046Z", 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min_mon_release": 19, 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "election_strategy": 1, 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": false, 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_ranks: ": "", 2024-09-15T21:31:43.577 INFO:teuthology.orchestra.run.smithi019.stdout: "features": { 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "persistent": [ 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.578 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "optional": [] 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "mons": [ 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 0, 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "a", 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.579 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:3300", 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6789", 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.580 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6789/0", 2024-09-15T21:31:43.581 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6789/0", 2024-09-15T21:31:43.581 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.581 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.581 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.581 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.581 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 1, 2024-09-15T21:31:43.583 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "b", 2024-09-15T21:31:43.583 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:3300", 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6789", 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.584 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6789/0", 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6789/0", 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 2, 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "c", 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.585 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:3301", 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6790", 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.586 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6790/0", 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6790/0", 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "feature_map": { 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": [ 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.587 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 1 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": [ 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 3 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: "client": [ 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.588 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 3 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": [ 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 1 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": false 2024-09-15T21:31:43.589 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:43.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-15T21:31:43.628 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:43.628 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:43.628 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:43.626+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:43.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:31:43.710 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "c", 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 2, 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "peon", 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: "election_epoch": 8, 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 36601, 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum": [ 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: 0, 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: 1, 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: 2 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.711 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_age": 31, 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "features": { 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "required_mon": [ 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.712 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_mon": [ 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.713 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.714 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.714 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.714 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.714 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.714 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.714 INFO:teuthology.orchestra.run.smithi019.stdout: "outside_quorum": [], 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "extra_probe_peers": [], 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "sync_provider": [], 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "monmap": { 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 1, 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2024-09-15T21:30:32.924046Z", 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2024-09-15T21:30:32.924046Z", 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min_mon_release": 19, 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "election_strategy": 1, 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": false, 2024-09-15T21:31:43.715 INFO:teuthology.orchestra.run.smithi019.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_ranks: ": "", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "features": { 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "persistent": [ 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.716 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.717 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.717 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.717 INFO:teuthology.orchestra.run.smithi019.stdout: "optional": [] 2024-09-15T21:31:43.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.717 INFO:teuthology.orchestra.run.smithi019.stdout: "mons": [ 2024-09-15T21:31:43.717 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.717 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 0, 2024-09-15T21:31:43.717 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "a", 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:3300", 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6789", 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6789/0", 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6789/0", 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 1, 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "b", 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.719 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.720 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:3300", 2024-09-15T21:31:43.720 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.720 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.720 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.720 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6789", 2024-09-15T21:31:43.720 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6789/0", 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6789/0", 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 2, 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "c", 2024-09-15T21:31:43.721 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:3301", 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6790", 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6790/0", 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6790/0", 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "feature_map": { 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": [ 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.723 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 1 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": [ 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 1 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: "client": [ 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.724 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.725 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.725 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 2 2024-09-15T21:31:43.725 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.725 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.725 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": false 2024-09-15T21:31:43.725 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:43.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-15T21:31:43.728 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:43.726+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:43.787 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:43.787 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:43.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:31:43.841 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:43.839+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "b", 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 1, 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: "state": "peon", 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: "election_epoch": 8, 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: "uptime": 36743, 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum": [ 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: 0, 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: 1, 2024-09-15T21:31:43.885 INFO:teuthology.orchestra.run.smithi019.stdout: 2 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_age": 31, 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "features": { 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "required_con": "2450521708007866372", 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "required_mon": [ 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.886 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_con": "4540719139924082687", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "quorum_mon": [ 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.887 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: "outside_quorum": [], 2024-09-15T21:31:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: "extra_probe_peers": [], 2024-09-15T21:31:43.888 INFO:teuthology.orchestra.run.smithi019.stdout: "sync_provider": [], 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "monmap": { 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 1, 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2024-09-15T21:30:32.924046Z", 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2024-09-15T21:30:32.924046Z", 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min_mon_release": 19, 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "min_mon_release_name": "squid", 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "election_strategy": 1, 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "disallowed_leaders: ": "", 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": false, 2024-09-15T21:31:43.889 INFO:teuthology.orchestra.run.smithi019.stdout: "tiebreaker_mon": "", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_ranks: ": "", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "features": { 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "persistent": [ 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "kraken", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "luminous", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "mimic", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "osdmap-prune", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "nautilus", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "octopus", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "pacific", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "elector-pinging", 2024-09-15T21:31:43.890 INFO:teuthology.orchestra.run.smithi019.stdout: "quincy", 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "reef", 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "squid" 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "optional": [] 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "mons": [ 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 0, 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "a", 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.891 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:3300", 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6789", 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6789/0", 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6789/0", 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.892 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 1, 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "b", 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:3300", 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.893 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6789", 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6789/0", 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6789/0", 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.894 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "rank": 2, 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "name": "c", 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:3301", 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:31:43.895 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6790", 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 0 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6790/0", 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6790/0", 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: "priority": 0, 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 0, 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_location": "{}" 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.896 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "feature_map": { 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "mon": [ 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 1 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": [ 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 4 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: "client": [ 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 4 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": [ 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: "features": "0x3f03dffffffdffff", 2024-09-15T21:31:43.898 INFO:teuthology.orchestra.run.smithi019.stdout: "release": "squid", 2024-09-15T21:31:43.899 INFO:teuthology.orchestra.run.smithi019.stdout: "num": 1 2024-09-15T21:31:43.899 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:43.899 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:43.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:43.899 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": false 2024-09-15T21:31:43.899 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:43.900 INFO:tasks.mon_recovery:restarting each monitor in turn 2024-09-15T21:31:43.900 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:31:43.900 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:31:43.912 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:31:43.912 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-09-15T21:31:43.912 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:43.915 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:43.915 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:43.924 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:31:43.965 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:43.963+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:44.032 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:44.029+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:44.067 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:44.067 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:44.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:31:44.160 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:44.158+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:44.219 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:44.219 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:44.228 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:31:44.233 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:44.231+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:44.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:31:44.319 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":31,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:44.333 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:44.331+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:44.361 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:31:44.362 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:31:44.373 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:44.373 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:44.378 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:31:44.378 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:31:44.382 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:31:44.394 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:31:44.394 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:31:44.411 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:31:44.412 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:31:44.428 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:31:44.429 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:31:44.434 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:44.432+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:44.445 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:31:44.445 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:31:44.461 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:31:44.461 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:31:44.478 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:31:44.478 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:31:44.486 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:31:44.486 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:31:44.486 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:31:44.486 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:31:44.487 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:31:44.495 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:31:44.535 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:44.533+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:44.597 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.143016533000001, 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:31:44.598 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:44.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:31:44.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:31:44.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:44.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:31:44.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:31:44.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:31:44.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:31:44.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:44.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:31:44.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:31:44.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:31:44.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:31:44.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:31:44.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:44.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:31:44.607 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.136876902999999, 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:31:44.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:44.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:31:44.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:31:44.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:31:44.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.133708727, 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.614 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:31:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:31:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:31:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:31:44.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:31:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:31:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.133162973999999, 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:31:44.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:31:44.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:44.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:44.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:31:44.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:31:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.624 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:31:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:31:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.140524665999999, 2024-09-15T21:31:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:31:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:44.625 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:31:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:31:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:31:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:31:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:31:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:31:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:31:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:31:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:31:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:44.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:31:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:31:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:31:44.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:31:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:31:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:31:44.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:31:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:31:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:31:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:31:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:31:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:31:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:31:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:31:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:31:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:31:44.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:44.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.137231838, 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:31:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:31:44.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 11.134681068000001, 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:44.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:44.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:31:44.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:31:44.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:44.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 11.131280379, 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:31:44.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:44.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:31:44.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:44.654 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:31:44.656 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:31:44.656 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:44.654+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:44.757 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:44.755+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:31:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:31:44.759 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:31:44.759 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:31:44.767 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:31:44.858 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:44.856+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:44.910 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:31:44.910 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:31:44.910 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:31:44.911 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:31:44.911 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:31:44.919 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:31:44.963 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:44.961+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:45.022 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:31:45.022 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:31:45.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:31:45.022 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:31:45.022 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:31:45.030 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:31:45.060 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:45.058+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:45.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:31:45.172 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:31:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:31:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:31:45.173 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:31:45.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:31:45.187 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:45.185+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:45.261 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:45.259+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:45.283 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:31:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:31:45.283 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:31:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:31:45.284 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:31:45.292 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:31:45.363 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:45.361+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:45.438 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:31:45.438 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:31:45.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:31:45.438 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:31:45.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.438 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:31:45.438 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.016901006000001, 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:31:45.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:31:45.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:31:45.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:31:45.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:31:45.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:31:45.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.993556160000001, 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.447 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:31:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:31:45.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.892189176999999, 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.453 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:31:45.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:31:45.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:31:45.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:31:45.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.892005358999999, 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.460 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:31:45.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:31:45.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:31:45.466 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891966514, 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:31:45.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:31:45.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.472 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891914539999998, 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:31:45.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:31:45.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:31:45.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:31:45.479 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891898812000001, 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:31:45.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:31:45.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891880164, 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.486 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:31:45.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:31:45.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891822213000001, 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:31:45.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:31:45.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:31:45.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891804382, 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:31:45.498 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:31:45.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:31:45.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891723356, 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:31:45.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:31:45.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:31:45.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:31:45.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891622100999999, 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:31:45.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:31:45.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.891575472, 2024-09-15T21:31:45.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:31:45.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:31:45.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:31:45.523 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.434231618999998, 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:31:45.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:31:45.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.529 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.434162429000001, 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:31:45.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:31:45.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.433897172999998, 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.536 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:31:45.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.433822902999999, 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:31:45.542 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:31:45.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:31:45.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.433794001999999, 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:31:45.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:31:45.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:31:45.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.433778673999999, 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:31:45.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:31:45.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 18.433694385999999, 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:31:45.561 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:31:45.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:31:45.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.566 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:31:45.567 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:31:45.567 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:31:45.568 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:45.566+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:45.569 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:31:45.669 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:45.668+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:45.676 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:45.676 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:45.676 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:31:45.676 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:45.685 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:31:45.771 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:45.769+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:45.792 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:45.792 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:45.792 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:31:45.792 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:31:45.792 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:45.800 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:31:45.886 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:45.884+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:45.950 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:45.950 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:31:45.950 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.506556962000001, 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.951 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:31:45.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.955 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.505643930000002, 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:31:45.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:31:45.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.960 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.399775657999999, 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.965 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.399363388000001, 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:31:45.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:31:45.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.398915218999999, 2024-09-15T21:31:45.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.398076871000001, 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:31:45.975 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:31:45.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:31:45.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:31:45.979 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.397160405000001, 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:31:45.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.946732834999999, 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:31:45.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:31:45.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:31:45.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.946286076, 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:31:45.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:31:45.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:31:45.994 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.945858833999999, 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:31:45.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:31:45.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:31:45.999 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.945391884999999, 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:31:46.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:31:46.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.945023341999999, 2024-09-15T21:31:46.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:31:46.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.944606272000001, 2024-09-15T21:31:46.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:31:46.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:31:46.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.941955256, 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:31:46.014 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:31:46.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.941257926999999, 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:31:46.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.657933032999999, 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.024 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.026 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:31:46.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.653943103, 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.029 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:31:46.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:31:46.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.653286841, 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.034 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:31:46.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:31:46.038 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.648833282999998, 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.039 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:31:46.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:31:46.042 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:31:46.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.043 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.043 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:31:46.044 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:31:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 18.646100528000002, 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.046 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:31:46.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.050 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:46.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:31:46.052 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:46.050+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:46.152 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:46.150+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:46.200 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:46.200 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:46.200 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:31:46.200 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:46.209 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:31:46.293 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:46.291+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:46.354 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:46.352+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:46.356 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:46.357 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:46.357 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:31:46.357 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:31:46.357 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:46.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:31:46.454 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:46.452+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:46.489 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 14.055132407, 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:31:46.514 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:31:46.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:31:46.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 13.049251014999999, 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:31:46.519 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:31:46.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:46.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:46.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:46.524 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:46.525 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:31:46.566 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:46.564+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:46.615 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:31:46.616 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:31:46.633 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:31:46.633 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:31:46.651 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:31:46.652 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:31:46.655 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:46.653+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:46.668 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:31:46.668 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:31:46.676 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:46.676 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:46.685 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:31:46.686 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:31:46.686 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:31:46.703 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:31:46.704 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:31:46.721 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:31:46.722 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:31:46.739 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:31:46.739 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:31:46.755 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:46.753+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:46.793 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:46.793 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:46.802 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:31:46.856 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:46.854+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:46.910 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:46.910 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:46.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:31:46.960 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:46.958+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:47.058 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:47.056+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:47.066 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:47.075 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:31:47.159 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:47.157+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:47.222 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:47.222 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:47.231 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:31:47.272 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:47.270+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:47.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:47.377 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:47.375+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:47.380 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:47.380 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:47.389 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:31:47.460 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:47.458+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:47.505 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:47.505 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:47.514 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:31:47.561 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:47.559+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:47.659 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:47.659 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:47.661 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:47.659+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:47.667 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:31:47.705 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":35,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:47.763 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:47.761+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:47.810 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:47.810 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:47.818 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:31:47.904 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:47.902+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:47.962 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:47.963 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:47.965 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:47.963+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:47.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:31:48.066 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:48.064+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:48.114 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:48.114 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:48.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:31:48.167 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:48.165+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:48.224 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:48.224 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:48.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:31:48.274 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:48.272+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:48.369 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:48.367+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:48.376 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:48.376 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:48.385 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:31:48.470 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:48.468+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:48.487 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:48.488 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:48.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:31:48.571 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:48.569+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:48.599 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:48.599 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:48.608 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:31:48.692 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:48.690+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:48.750 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:48.750 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:48.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:31:48.759 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:31:48.796 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:48.794+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:48.873 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:48.871+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:48.904 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:48.904 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:48.912 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:31:48.975 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:48.973+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:49.015 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:49.015 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:49.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:31:49.075 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:49.074+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:49.136 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:49.136 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:49.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:31:49.189 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:49.187+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:49.252 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:49.252 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:49.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:31:49.277 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:49.276+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:49.378 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:49.376+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:49.408 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:49.409 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:49.417 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:31:49.479 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:49.477+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:49.524 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:49.525 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:49.533 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:31:49.579 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:49.577+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:49.641 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:49.641 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:49.649 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:31:49.680 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:49.678+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:49.756 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:49.757 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:49.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:31:49.809 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:49.807+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:49.872 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:49.872 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:49.881 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:31:49.882 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:49.881+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:49.983 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:49.982+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:50.028 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:50.029 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:50.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:31:50.084 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:50.082+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:50.145 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:50.146 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:50.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:31:50.185 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:50.183+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:50.261 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:50.261 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:50.270 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:31:50.313 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:50.311+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:50.376 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:50.376 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:50.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:31:50.426 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:50.424+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:50.488 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:50.486+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:50.532 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:50.532 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:50.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:31:50.588 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:50.586+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:50.656 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:50.656 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:50.666 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:31:50.689 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:50.688+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:50.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:50.790 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:50.788+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:50.809 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:50.809 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:50.818 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:31:50.903 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:50.901+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:50.962 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:50.962 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:50.971 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:31:50.992 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:50.990+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:51.093 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":38,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:51.093 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:51.091+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:51.114 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:51.114 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:51.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:31:51.194 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:51.192+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:51.225 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:51.225 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:51.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:31:51.296 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:51.294+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:51.336 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:51.336 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:51.345 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:31:51.397 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:51.395+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:51.447 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:51.447 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:51.456 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:31:51.498 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:51.495+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:51.598 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:51.596+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:51.598 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:51.599 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:51.607 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:31:51.699 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:51.697+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:51.750 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:51.750 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:51.760 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:31:51.804 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:51.802+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:51.863 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:51.863 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:51.872 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:31:51.900 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:51.899+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:52.001 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:51.999+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:52.014 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:52.014 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:52.023 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:31:52.107 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:52.105+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:52.166 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:52.166 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:52.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:31:52.216 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:52.214+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:52.294 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:52.295 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:52.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:31:52.305 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:52.304+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:52.406 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:52.404+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:52.412 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:52.412 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:52.421 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:31:52.507 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:52.505+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:52.528 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:52.528 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:52.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:31:52.621 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:52.619+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:52.685 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:52.685 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:52.693 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:31:52.709 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:52.707+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:52.812 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:52.810+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:52.839 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:52.839 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:52.848 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:31:52.913 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:52.911+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:52.955 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:52.955 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:52.964 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:31:53.013 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:53.012+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:53.072 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:53.072 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:53.081 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:31:53.114 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:53.112+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:53.188 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:53.188 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:53.197 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:31:53.240 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:53.238+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:53.303 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:53.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:31:53.353 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:53.351+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:53.416 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:53.414+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:53.460 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:53.461 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:53.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:31:53.553 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:53.551+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:53.616 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:53.616 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:53.618 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:53.616+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:53.625 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:31:53.719 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:53.718+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:53.773 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:53.773 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:53.782 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:31:53.826 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:53.824+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:53.898 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:53.898 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:53.907 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:31:53.922 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:53.920+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:54.023 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:54.021+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:54.051 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:54.051 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:54.060 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:31:54.106 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:54.150 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:54.148+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:54.203 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:54.203 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:54.211 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:31:54.255 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:54.253+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:54.314 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:54.314 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:54.323 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:31:54.364 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:54.363+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:54.426 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:54.424+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:54.466 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:54.466 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:54.475 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:31:54.527 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:54.525+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:54.618 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:54.618 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:54.627 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:31:54.628 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:54.625+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:54.728 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:54.726+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:54.730 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:54.730 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:54.738 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:31:54.829 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:54.827+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:54.882 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:54.882 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:54.890 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:31:54.932 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:54.931+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:55.033 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:55.031+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:55.034 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:55.034 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:55.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:31:55.133 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:55.131+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:55.147 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:55.147 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:55.156 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:31:55.241 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:55.240+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:55.299 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:55.299 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:55.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:31:55.349 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:55.348+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:55.436 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:55.434+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:55.450 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:55.451 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:55.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:31:55.537 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:55.536+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:55.579 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:55.579 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:55.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:31:55.638 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:55.637+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:55.696 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:55.696 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:55.705 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:31:55.749 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:55.747+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:55.812 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:55.812 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:55.821 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:31:55.862 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:55.860+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:55.939 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:55.938+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:55.968 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:55.968 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:55.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:31:56.040 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:56.038+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:56.083 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:56.084 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:56.093 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:31:56.177 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:56.175+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:56.241 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:56.241 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:56.241 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:56.239+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:56.250 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:31:56.342 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:56.341+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:56.398 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:56.398 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:56.407 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:31:56.442 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:56.441+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:56.513 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:56.513 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:56.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:31:56.567 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:56.565+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:56.630 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:56.630 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:56.639 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:31:56.680 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:56.678+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:56.745 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:56.743+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:56.787 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:56.787 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:56.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:31:56.881 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:56.879+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:56.943 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:56.943 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:56.946 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:56.944+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:56.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:31:57.048 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:57.045+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:57.099 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:57.099 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:57.108 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:31:57.152 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:57.150+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:57.223 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:57.223 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:57.232 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:31:57.274 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:57.271+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:57.350 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:57.349+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:57.377 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:57.378 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:57.386 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:31:57.451 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:57.449+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:57.488 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:57.489 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:57.498 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:31:57.552 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:57.550+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:57.642 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:57.643 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:57.652 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:31:57.653 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:57.652+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:57.754 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:57.754 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:57.755 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:57.752+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:57.762 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:31:57.855 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:57.853+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:57.905 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:57.905 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:57.914 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:31:57.956 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:57.954+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:58.057 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:58.054+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:58.057 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:58.057 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:58.066 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:31:58.158 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:31:58.156+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:31:58.209 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:58.210 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:58.218 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:31:58.259 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:58.257+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:58.360 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:58.358+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:58.361 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:58.361 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:58.370 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:31:58.461 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:58.459+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:58.473 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:58.473 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:58.481 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:31:58.561 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:31:58.559+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:31:58.584 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:58.584 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:58.593 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:31:58.663 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:58.661+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:58.724 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:31:58.725 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:31:58.737 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:31:58.737 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:31:58.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:31:58.757 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:31:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:31:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:set 2.2 pg_upmap mapping to [6,5] 2024-09-15T21:31:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:31:58.763 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:58.761+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:58.777 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:31:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:31:58.811 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:31:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:31:58.836 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:31:58.836 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:31:58.865 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:58.863+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:58.871 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:31:58.872 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:31:58.874 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:58.874 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:58.874 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:31:58.875 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:58.886 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:31:58.894 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:31:58.894 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:31:58.911 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:31:58.911 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:31:58.966 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:31:58.964+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:31:59.002 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:59.002 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:59.002 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:31:59.002 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:31:59.002 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:59.010 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:31:59.033 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":10,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:31:59.047 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-09-15T21:31:59.048 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:31:59.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:31:59.089 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:31:59.087+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:31:59.090 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:31:59.090 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-09-15T21:31:59.091 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:31:59.145 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:59.145 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:31:59.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:31:59.145 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:31:59.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.145 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.700520832000002, 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:31:59.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:31:59.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.699607800000003, 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:31:59.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.593739528, 2024-09-15T21:31:59.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.159 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.593327258000002, 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.160 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:31:59.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.164 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.592879089, 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:31:59.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:31:59.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.169 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.592040740999998, 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.170 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:31:59.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.591124274999999, 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.175 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:31:59.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:31:59.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:31:59.179 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.140696705000003, 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:31:59.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:31:59.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.140249945999997, 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.188 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:31:59.189 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.139822703999997, 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:31:59.190 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.194 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.139355754999997, 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.196 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.197 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:31:59.198 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.199 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.138987212000004, 2024-09-15T21:31:59.200 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:31:59.201 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.202 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:31:59.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.204 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.138570141999999, 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.205 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.206 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.208 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.209 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.135919125999997, 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:31:59.210 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:31:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:31:59.213 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:31:59.214 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 32.135221797, 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.215 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:31:59.216 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.217 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 31.851896903, 2024-09-15T21:31:59.219 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:31:59.220 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:31:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:31:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:31:59.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 31.847906973000001, 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:31:59.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.227 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:31:59.228 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:31:59.229 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 31.847250711000001, 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:31:59.230 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.231 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:31:59.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.233 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 31.842797152999999, 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.236 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.237 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.238 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 31.840064397999999, 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.239 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:31:59.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:31:59.241 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.242 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.243 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:59.245 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:31:59.245 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:59.242+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:59.346 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:59.344+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:59.353 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:59.353 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:59.353 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:31:59.354 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:59.362 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:31:59.447 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:31:59.445+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:31:59.471 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:59.471 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:59.472 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:31:59.472 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:31:59.472 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:59.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:31:59.548 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:59.546+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:59.627 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:59.627 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:31:59.627 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:31:59.627 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:31:59.627 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 27.168562156, 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:31:59.628 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.629 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:31:59.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.631 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.632 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 26.162680764000001, 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:31:59.633 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.634 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:31:59.635 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.636 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:31:59.637 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:31:59.638 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:59.639 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:31:59.648 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:31:59.646+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:31:59.746 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:59.746 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:59.746 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:31:59.746 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:59.749 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:59.747+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:31:59.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:31:59.850 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:31:59.848+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:31:59.861 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:31:59.862 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:31:59.862 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:31:59.862 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:31:59.862 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:31:59.870 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:31:59.951 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:31:59.949+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 28.560411059, 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:32:00.018 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:00.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:32:00.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:00.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:00.023 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:00.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:00.052 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:00.050+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:00.134 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:00.134 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:00.134 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:00.134 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:00.142 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:00.186 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:00.184+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:00.250 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:00.251 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:00.251 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:00.251 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:00.251 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:00.254 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:00.252+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:00.258 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:00.355 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:00.353+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:00.407 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:00.407 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:00.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:00.407 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [] 2024-09-15T21:32:00.407 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:00.416 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:00.460 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:00.458+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:00.531 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:00.531 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:00.531 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:00.540 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:00.557 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:00.556+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:00.658 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:00.656+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:00.685 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:00.685 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:00.685 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:00.685 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:00.685 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:00.694 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:00.778 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:00.776+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:00.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:32:00.836 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:00.837 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:00.837 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:00.845 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:00.860 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:00.858+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:00.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:32:00.968 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:00.966+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:00.987 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:00.987 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:00.988 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:00.996 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:01.062 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:01.060+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:01.099 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:01.099 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:01.099 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:01.099 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:01.100 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:01.108 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:01.163 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:01.161+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:01.252 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:01.252 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:01.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:01.252 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:01.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.252 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 29.797551057, 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:01.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:01.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:01.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:01.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 29.791411427, 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:01.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:32:01.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 29.788243251000001, 2024-09-15T21:32:01.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:01.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:32:01.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:32:01.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:32:01.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 29.787697498, 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:01.269 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:01.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:32:01.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.79505919, 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:01.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:01.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:32:01.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:01.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:32:01.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 28.791766362000001, 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:01.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:32:01.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:32:01.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:01.285 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.789215592000001, 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:01.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:32:01.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:32:01.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:01.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.785814902999999, 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:01.292 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:32:01.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:01.296 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:01.297 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:01.298 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:01.299 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:01.296+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:01.399 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:01.397+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:01.441 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:01.441 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:01.442 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:01.442 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:01.450 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:01.500 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:01.498+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:01.553 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:01.553 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:01.553 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:01.553 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:01.553 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:01.561 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:01.605 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:01.603+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:01.664 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:01.665 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:01.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:01.665 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:01.665 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:01.673 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:01.701 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:01.700+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:01.802 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:01.800+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:01.817 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:01.817 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:01.818 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:01.818 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:01.826 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:01.903 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:01.902+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:01.969 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:01.969 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:01.969 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:01.969 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:01.969 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:01.978 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:02.005 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:02.004+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.661096274999998, 2024-09-15T21:32:02.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:02.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:32:02.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:32:02.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.637751428999998, 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:32:02.089 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:32:02.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:32:02.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:02.095 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.536384446, 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:32:02.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.536200628000003, 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.102 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:32:02.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:32:02.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:32:02.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.536161782999997, 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:32:02.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:32:02.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:32:02.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.536109809000003, 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.114 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:32:02.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:32:02.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.120 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.536094081000002, 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:32:02.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.536075433000001, 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:32:02.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:32:02.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.536017481999998, 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:32:02.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:32:02.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:32:02.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.535999650999997, 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:32:02.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:32:02.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:32:02.144 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.535918625000001, 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.145 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:32:02.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:32:02.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:32:02.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.535817369999997, 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.151 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:02.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:32:02.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.535770741, 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:32:02.157 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:32:02.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:32:02.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.078426888000003, 2024-09-15T21:32:02.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:32:02.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.169 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.078357697999998, 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:32:02.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:32:02.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.175 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.078092441999999, 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.176 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:32:02.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:32:02.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.078018172, 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.182 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:02.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:32:02.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:32:02.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.077989271, 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.188 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:02.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:32:02.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.077973943000003, 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.194 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:32:02.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.077889655, 2024-09-15T21:32:02.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:02.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:32:02.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:02.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:02.206 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:02.207 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:02.208 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:02.209 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:02.206+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:02.216 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-09-15T21:32:02.216 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:32:02.216 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:32:02.227 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:02.228 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:02.240 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:32:02.241 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-09-15T21:32:02.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:02.252 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:02.252 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:02.271 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:02.271 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:02.289 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:02.289 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:02.306 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:02.307 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:02.308 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:02.306+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:02.324 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:02.324 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:02.332 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:02.332 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:02.341 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:02.341 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:02.342 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:02.366 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:02.366 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:02.424 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:02.422+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:02.477 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:02.477 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:02.486 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:02.527 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:02.525+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:02.610 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:02.608+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:02.634 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:02.634 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:02.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:02.644 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:02.711 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:02.709+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:02.751 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:02.752 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:02.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:02.812 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:02.810+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:02.867 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:02.868 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:02.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:02.913 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:02.911+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:02.983 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:02.983 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:02.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:03.014 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:03.012+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:03.099 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:03.099 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:03.109 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:03.153 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:03.151+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:03.215 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:03.213+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:03.216 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:03.216 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:03.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:03.316 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:03.314+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:03.372 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:03.372 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:03.381 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:03.422 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:03.420+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:03.518 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:03.515+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:03.528 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:03.529 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:03.537 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:03.619 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:03.617+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:03.644 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:03.644 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:03.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:03.737 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:03.735+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:03.782 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:32:03.782 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:32:03.783 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T21:32:03.783 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 0.870335 2024-09-15T21:32:03.783 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 0.8703348682367198 2024-09-15T21:32:03.800 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:03.801 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:03.810 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:03.854 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:03.852+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:03.921 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:03.918+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:03.926 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:03.926 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:03.935 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:04.022 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:04.020+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:04.080 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:04.080 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:04.089 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:04.123 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:04.122+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:04.191 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:04.192 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:04.200 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:04.245 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:04.243+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:04.303 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:04.303 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:04.312 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:04.353 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:04.351+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:04.383 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:32:04.428 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:04.426+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:04.456 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:04.456 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:04.463 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:04.510 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:04.511 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:04.527 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:04.525+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:04.527 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:04.527 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:04.543 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:04.543 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:04.559 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:04.559 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:04.567 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:04.567 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:04.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:04.576 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:04.577 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:04.593 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:04.593 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:04.612 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:04.612 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:04.628 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:04.626+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:04.628 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:04.628 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:04.680 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:04.680 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:04.681 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:04.689 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:04.728 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:04.726+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:04.829 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:04.827+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:04.832 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:04.832 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:04.832 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:04.832 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:04.832 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:04.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:04.930 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:04.928+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:04.984 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:04.984 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:04.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:04.985 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:04.985 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:04.993 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:05.031 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:05.029+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:05.132 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:05.130+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:05.137 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:05.137 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:05.137 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:05.146 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:05.233 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:05.230+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:05.289 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:05.289 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:05.289 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:05.289 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:05.289 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:05.298 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:05.339 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:05.337+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:05.435 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:05.434+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 40.022236736000004, 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:32:05.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:32:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.998891890000003, 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:32:05.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.457 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897524906999998, 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:32:05.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897341089000001, 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:32:05.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:32:05.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.470 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897302244000002, 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:32:05.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897250270000001, 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:32:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:05.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:32:05.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:32:05.485 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897234542, 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:32:05.486 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:32:05.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:32:05.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897215893999999, 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897157943000003, 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.499 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897140112000002, 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:32:05.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:32:05.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.897059085999999, 2024-09-15T21:32:05.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:32:05.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:32:05.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.896957831000002, 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.519 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:32:05.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.525 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.896911201999998, 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:32:05.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:32:05.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.531 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.439567349000001, 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.532 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:32:05.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:32:05.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.439498159000003, 2024-09-15T21:32:05.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.539 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:05.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:32:05.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:32:05.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:05.545 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.439232902999997, 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:32:05.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:32:05.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.439158632999998, 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:32:05.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:32:05.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:32:05.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.439129731999998, 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.558 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:32:05.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.564 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.439114404000001, 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:32:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:32:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 38.439030115999998, 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:32:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:05.577 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:05.579 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:05.580 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:05.577+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:05.659 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:05.680 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:05.677+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:05.700 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:05.700 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:05.701 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:05.701 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:05.711 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:05.781 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:05.779+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:05.860 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:05.860 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:05.861 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:05.861 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:05.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:05.882 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:05.880+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 39.533988434999998, 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:05.978 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:05.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:32:05.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:32:05.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:05.983 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 39.533075402999998, 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:05.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:32:05.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:32:05.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:05.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:32:05.988 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 39.427207131000003, 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:05.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:05.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:32:05.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:32:05.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:05.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 39.426794860999998, 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:05.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:32:05.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:05.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:05.999 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 39.426346692000003, 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.004 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 39.425508344000001, 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:32:06.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 39.424591878000001, 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.009 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:32:06.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:32:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:32:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:32:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:32:06.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.014 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.974164307999999, 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:32:06.019 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.973717549, 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:06.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:32:06.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.973290306999999, 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:06.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.026 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:32:06.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.029 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.972823357999999, 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:32:06.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.972454814999999, 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:32:06.034 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.036 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:32:06.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.972037745000002, 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:32:06.039 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:32:06.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.043 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.969386729, 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.047 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.968689400000002, 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.049 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.050 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.051 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:32:06.052 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.053 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.685364505999999, 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.054 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:32:06.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.056 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:32:06.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.681374576000003, 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:32:06.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:32:06.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:06.063 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.680718314000003, 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.064 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.065 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:32:06.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.067 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.676264756000002, 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.068 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.069 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:32:06.070 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.071 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:32:06.072 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 38.673532000999998, 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.073 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:32:06.074 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.075 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:32:06.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.077 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.078 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.078 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:06.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:06.080 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:06.081 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:06.077+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:06.179 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:06.177+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:06.228 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:06.228 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:06.228 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:06.237 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:06.280 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:06.278+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:06.380 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:06.378+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:06.384 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:06.384 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:06.385 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:06.393 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:06.482 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:06.480+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:06.540 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:06.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 34.081524530999999, 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.541 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:06.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:32:06.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 33.075643139, 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:06.546 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.547 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:06.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.549 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:06.550 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:06.552 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:06.593 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:06.591+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:06.640 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:32:06.684 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:06.682+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:06.700 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:06.700 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:06.701 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:06.709 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:06.766 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:06.766 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:06.785 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:06.783+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:06.785 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:06.785 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:06.801 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:06.802 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:06.820 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:06.821 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:06.822 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:06.823 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:06.832 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:06.838 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:06.838 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:06.856 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:06.857 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:06.874 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:06.874 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:06.885 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:06.883+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:06.892 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:06.892 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:06.946 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:06.946 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:06.954 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:06.998 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:06.996+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:07.062 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:07.062 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:07.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:07.113 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:07.110+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:07.198 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:07.196+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:07.219 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:07.219 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:07.228 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:07.312 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:07.310+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:07.375 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:07.375 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:07.384 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:07.428 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:07.426+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:07.491 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:07.489+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:07.500 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:07.500 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:07.509 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:07.593 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:07.591+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:07.654 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:07.654 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:07.663 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:07.705 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:07.703+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:07.793 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:07.791+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:07.807 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:07.807 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:07.815 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:07.893 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:07.892+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:07.958 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:07.958 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:07.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:07.995 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:07.993+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:08.069 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:08.069 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:08.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:08.122 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:08.120+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:08.180 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:08.181 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:08.189 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:08.199 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:08.196+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:08.300 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:08.298+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:08.336 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:08.337 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:08.342 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:08.401 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:08.399+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:08.485 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:08.485 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:08.493 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:08.502 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:08.500+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:08.596 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:08.596 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:08.602 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:08.600+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:08.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:08.703 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:08.701+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:08.707 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:08.707 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:08.716 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:08.804 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:08.802+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:08.818 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:08.818 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:08.827 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:08.904 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:32:08.905 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:08.902+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:08.970 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:08.970 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:08.980 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:09.025 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:09.022+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:09.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:09.093 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:09.093 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:09.102 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:09.144 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:09.141+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:09.208 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:09.206+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:09.252 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:09.252 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:09.261 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:09.308 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:09.306+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:09.369 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:09.369 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:09.378 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:09.409 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:09.407+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:09.455 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:09.510 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:09.507+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:09.526 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:09.527 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:09.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:09.620 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:09.617+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:09.683 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:09.683 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:09.692 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:09.733 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:09.730+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:09.812 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:09.810+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:09.840 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:09.840 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:09.849 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:09.913 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:09.911+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:09.957 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:09.957 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:09.966 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:10.014 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:10.012+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:10.073 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:10.073 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:10.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:10.126 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:10.123+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:10.189 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:10.189 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:10.198 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:10.239 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:10.236+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:10.318 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:10.315+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:10.346 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:10.346 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:10.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:10.419 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:10.416+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:10.461 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:10.462 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:10.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:10.519 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:10.516+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:10.578 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:10.578 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:10.587 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:10.631 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:10.629+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:10.703 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:10.703 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:10.712 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:10.721 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:10.718+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:10.822 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:10.820+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:10.855 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:10.855 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:10.864 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:10.923 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:10.921+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:10.967 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:10.967 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:10.975 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:11.059 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:11.057+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:11.118 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:11.119 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:11.125 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:11.123+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:11.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:11.226 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:11.223+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:11.270 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:11.270 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:11.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:11.327 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:11.324+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:11.382 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:11.382 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:11.391 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:11.434 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:11.432+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:11.494 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:11.494 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:11.502 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:11.543 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:11.541+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:11.629 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:11.626+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:11.645 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:11.645 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:11.654 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:11.730 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:11.728+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:11.757 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:11.757 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:11.766 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:11.831 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:11.829+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:11.910 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:11.910 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:11.918 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:11.932 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:11.930+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:12.022 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:12.022 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:12.031 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:12.033 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:12.029+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:12.133 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:12.131+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:12.135 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:12.135 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:12.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:12.234 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:12.231+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:12.264 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:12.264 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:12.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:12.336 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:12.334+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:12.424 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:12.431 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:12.436 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:12.434+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:12.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:12.537 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:12.534+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:12.541 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:12.541 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:12.550 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:12.638 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:12.635+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:12.698 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:12.698 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:12.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:12.739 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:12.737+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:12.841 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:12.838+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:12.854 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:12.863 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:12.947 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:12.944+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:13.010 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:13.010 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:13.019 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:13.043 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:13.041+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:13.144 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:13.141+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:13.166 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:13.167 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:13.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:13.245 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:13.242+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:13.323 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:13.323 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:13.332 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:13.346 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:13.344+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:13.439 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:13.439 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:13.446 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:13.443+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:13.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:13.547 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:13.545+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:13.596 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:13.596 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:13.604 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:13.648 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:13.646+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:13.710 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:13.711 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:13.719 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:13.763 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:13.760+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:13.826 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:13.827 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:13.835 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:13.879 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:13.877+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:13.950 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:13.950 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:13.951 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:13.948+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:13.959 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:14.052 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:14.050+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:14.103 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:14.103 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:14.112 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:14.153 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:14.151+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:14.254 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:14.251+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:14.255 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:14.256 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:14.264 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:14.355 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:14.353+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:14.367 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:14.367 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:14.377 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:14.456 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:14.454+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:14.480 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:14.480 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:14.488 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:14.572 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:14.570+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:14.632 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:14.632 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:14.640 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:14.657 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:14.655+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:14.758 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:14.756+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:14.783 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:14.784 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:14.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:14.859 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:14.856+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:14.936 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:14.936 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:14.944 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:14.960 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:14.958+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:15.047 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:15.047 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:15.056 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:15.061 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:15.058+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:15.158 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:15.158 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:15.163 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:15.160+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:15.168 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:15.263 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:15.261+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:15.270 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:15.271 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:15.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:15.365 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:15.362+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:15.382 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:15.383 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:15.391 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:15.465 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:15.463+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:15.512 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:15.513 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:15.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:15.566 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:15.564+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:15.631 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:15.631 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:15.640 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:15.667 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:15.665+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:15.747 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:15.747 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:15.756 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:15.800 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:15.797+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:15.863 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:15.864 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:15.869 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:15.866+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:15.872 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:15.970 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:15.967+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:16.019 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:16.019 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:16.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:16.071 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:16.069+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:16.172 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:16.169+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:16.176 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:16.176 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:16.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:16.273 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:16.270+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:16.333 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:16.333 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:16.341 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:16.374 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:16.372+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:16.475 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:16.472+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:16.489 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:16.489 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:16.498 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:16.582 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:16.579+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:16.645 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:16.646 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:16.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:16.677 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:16.675+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:16.778 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:16.775+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:16.802 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:16.802 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:16.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:16.895 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:16.892+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:16.958 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:16.959 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:16.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:16.980 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:16.978+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:17.081 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:17.079+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:17.114 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:17.114 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:17.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:17.181 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:17.179+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:17.236 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:17.237 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:17.241 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:17.241 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:17.250 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:17.253 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:17.254 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:17.270 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:17.270 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:17.286 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:17.286 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:17.291 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:17.289+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:17.303 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:17.303 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:17.320 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:17.320 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:17.338 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:17.338 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:17.354 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:17.355 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:17.382 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:17.379+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:17.397 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:17.397 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:17.397 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:17.397 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:17.397 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:17.406 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:17.483 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:17.480+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:17.487 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:17.500 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-09-15T21:32:17.500 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:32:17.501 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:32:17.508 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:17.508 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:17.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:17.508 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:17.508 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:17.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:17.543 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:32:17.543 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-09-15T21:32:17.544 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:17.584 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:17.582+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:17.621 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:17.621 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:17.621 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:17.630 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:17.686 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:17.683+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:17.773 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:17.774 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:17.774 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:17.774 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:17.774 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:17.782 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:17.786 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:17.784+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.431407806999999, 2024-09-15T21:32:17.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:17.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:32:17.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:32:17.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:32:17.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:32:17.893 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.425268177, 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:32:17.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.422100000999997, 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:32:17.898 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:17.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.902 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 46.421554248, 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.903 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:17.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:32:17.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.428915940000003, 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:17.907 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:17.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:32:17.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:17.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:32:17.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.425623111999997, 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:17.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:32:17.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:32:17.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.423072341999998, 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:32:17.918 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:32:17.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:32:17.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:17.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:32:17.925 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.419671653000002, 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:32:17.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:17.930 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:17.930 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:17.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:17.932 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:17.929+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:18.032 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:18.030+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:18.034 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:18.035 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:18.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:18.133 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:18.131+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:18.186 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:18.186 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:18.186 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:18.187 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:18.187 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:18.195 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:18.234 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:18.232+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:18.336 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:18.334+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:18.338 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:18.338 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:18.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:18.338 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:18.338 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:18.347 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:18.437 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:18.435+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:18.490 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:18.491 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:18.499 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:18.540 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:18.538+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:18.639 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:18.637+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:18.643 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:18.643 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:18.643 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:18.652 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:18.739 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:18.737+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.33523993, 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:32:18.756 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:32:18.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:32:18.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:32:18.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:32:18.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:32:18.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.311895084, 2024-09-15T21:32:18.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:32:18.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:32:18.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:32:18.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:32:18.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210528101000001, 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.774 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:32:18.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:32:18.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210344282999998, 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.781 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:32:18.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210305437999999, 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.788 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.794 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210253463999997, 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.795 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:32:18.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.801 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210237736000003, 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:32:18.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:32:18.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210219088000002, 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.808 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:32:18.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:32:18.814 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210161137, 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.815 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:32:18.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:32:18.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210143305999999, 2024-09-15T21:32:18.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:32:18.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.210062280000002, 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.829 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:32:18.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:32:18.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.209961024999998, 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:32:18.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:32:18.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:32:18.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:32:18.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.209914396000002, 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.841 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:32:18.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.752570542999997, 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.848 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:32:18.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:18.854 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.752501353, 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:32:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:32:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:32:18.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:32:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:32:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:32:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:32:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:32:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.752236097000001, 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.863 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:32:18.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:32:18.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.752161827000002, 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.869 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:32:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:18.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:32:18.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:32:18.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:18.876 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.752132926000002, 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:32:18.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:32:18.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.752117597999998, 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.883 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:18.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:32:18.889 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.752033310000002, 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:18.890 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:18.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:18.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:32:18.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:18.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:32:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:18.898 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:18.895+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:18.898 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:18.999 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:18.997+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:19.018 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:19.019 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:19.019 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:19.019 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:19.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:19.099 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:19.097+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:19.136 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:19.136 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:19.136 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:19.136 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:19.136 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:19.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:19.200 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:19.197+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:19.253 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:19.253 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:19.253 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.809583691, 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.254 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:32:19.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:32:19.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.808670659000001, 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.259 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.260 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:32:19.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.262 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:32:19.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:32:19.264 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.702802386999998, 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.265 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:32:19.266 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.267 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:32:19.268 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.702390117, 2024-09-15T21:32:19.269 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:32:19.271 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.272 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.273 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.701941947999998, 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:32:19.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:32:19.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.701103600000003, 2024-09-15T21:32:19.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:32:19.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:32:19.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.700187133999997, 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.283 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:32:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:32:19.286 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.287 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.249759564000001, 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.288 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:19.289 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:32:19.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:32:19.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.292 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.249312805000002, 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:19.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:32:19.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.295 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:32:19.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:19.297 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.248885563000002, 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.298 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.299 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:32:19.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.301 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.248418614000002, 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.302 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:32:19.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.304 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.305 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:32:19.306 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.248050071000002, 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:32:19.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.309 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.310 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:32:19.311 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.247633000999997, 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.312 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:32:19.313 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:32:19.316 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.244981985000003, 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.317 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:32:19.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:32:19.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:32:19.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.244284655999998, 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.322 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 51.960959762000002, 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:32:19.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:32:19.330 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 51.956969831999999, 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:32:19.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 51.956313569999999, 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:19.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 51.951860011999997, 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:32:19.340 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:32:19.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 51.949127257000001, 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:32:19.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:32:19.349 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.350 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.350 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.350 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.350 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:19.351 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:19.352 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:19.349+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:19.367 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:32:19.451 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:19.449+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:19.502 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:19.502 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:19.511 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:19.552 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:19.550+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:19.619 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:19.619 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:19.619 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:19.619 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:19.619 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:19.628 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:19.652 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:19.651+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:19.735 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:19.736 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:19.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:19.736 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:19.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.736 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:32:19.736 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:19.736 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 47.276881545999998, 2024-09-15T21:32:19.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:19.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:32:19.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.740 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:32:19.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:32:19.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.741 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.741 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:32:19.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:32:19.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 46.271000153999999, 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:19.743 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:19.743 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:19.743 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:19.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:19.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:32:19.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:19.746 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.747 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:19.748 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:19.748 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:19.749 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:19.753 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:19.751+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:19.853 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:19.850+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:19.856 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:19.856 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:19.856 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:19.856 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:19.866 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:19.954 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:19.952+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:20.013 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:20.013 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:20.014 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:20.014 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:20.014 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:20.022 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:20.055 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:20.053+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:20.156 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:20.153+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:20.169 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:20.169 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:20.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:20.169 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:20.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.169 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:32:20.169 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:20.169 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 48.711587751000003, 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:20.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:32:20.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:32:20.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:20.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:20.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:20.175 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:20.175 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:20.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:20.257 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:20.255+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:20.325 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:20.325 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:20.325 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:20.325 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:20.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:20.377 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:20.375+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:20.440 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:20.441 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:20.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:20.459 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:20.457+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:20.560 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:20.558+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:20.596 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:20.596 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:20.596 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:20.596 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [] 2024-09-15T21:32:20.597 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:20.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:20.661 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:20.659+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:20.719 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:20.719 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:20.719 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:20.719 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:20.728 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:20.762 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:20.760+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:20.832 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:20.832 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:20.833 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:20.833 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:20.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:20.863 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:20.861+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:20.944 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:20.944 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:20.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:20.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:20.945 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:20.953 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:20.963 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:20.961+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:21.055 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:21.055 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:21.055 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:21.056 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:21.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:21.064 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:21.062+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:21.165 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:21.163+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:21.207 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:21.207 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:21.207 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:21.207 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:21.207 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:21.215 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:21.266 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:21.264+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.864202159000001, 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:32:21.319 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:21.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:32:21.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:32:21.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:21.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:32:21.326 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.858062529000001, 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:21.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:32:21.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.854894352999999, 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:21.332 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:32:21.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:32:21.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:21.336 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 49.854348600000002, 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:21.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:32:21.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.340 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.861710291999998, 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:21.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:21.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:32:21.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:21.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:32:21.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:21.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.858417463999999, 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:21.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:32:21.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.855866693999999, 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:21.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:32:21.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:21.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:32:21.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.852466004999997, 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:21.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:32:21.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:32:21.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:21.364 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:21.364 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:21.365 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:21.406 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:21.404+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:21.468 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:21.466+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:21.509 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:21.509 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:21.509 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:21.509 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:21.518 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:21.569 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:21.567+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:21.662 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:21.662 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:21.662 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:21.662 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:21.662 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:21.669 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:21.667+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:21.670 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:21.770 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:21.768+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:21.814 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:21.814 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:21.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:21.814 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:21.814 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:21.823 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:21.871 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:21.869+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:21.966 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:21.966 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:21.966 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:21.972 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:21.970+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:21.975 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:22.072 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:22.070+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:22.076 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:22.077 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:22.077 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:22.077 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:22.077 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:22.085 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:22.173 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:22.170+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:22.231 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:22.231 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:22.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:22.231 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:22.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.231 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.810236209000003, 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:32:22.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.238 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.786891363000002, 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:32:22.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685524379999997, 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.245 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:32:22.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685340562, 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.251 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:32:22.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:32:22.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685301717000002, 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.257 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:32:22.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685249743, 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.263 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:32:22.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:22.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.269 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685234014999999, 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:32:22.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685215366999998, 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:32:22.276 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:32:22.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:32:22.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685157416000003, 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.283 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:32:22.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:32:22.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.289 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685139585000002, 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:22.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:32:22.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:32:22.295 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.685058558999998, 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.296 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:22.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:32:22.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.684957304000001, 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.302 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:32:22.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:32:22.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.684910674999998, 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.309 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:32:22.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:32:22.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.315 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.227566822, 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:32:22.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:32:22.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:22.322 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.227497632000002, 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:22.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:32:22.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:32:22.328 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.227232376000003, 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.329 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:32:22.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.227158105999997, 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:22.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:32:22.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.227129204999997, 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:32:22.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:32:22.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.227113877000001, 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.347 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:32:22.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:32:22.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:32:22.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.227029588999997, 2024-09-15T21:32:22.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:22.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:32:22.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:32:22.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:22.360 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:22.360 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:22.361 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:22.359+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:22.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:22.462 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:22.460+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:22.481 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:22.482 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:22.482 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:22.482 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:22.491 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:22.562 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:22.561+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:22.599 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:22.599 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:22.599 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:22.599 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:22.599 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:22.608 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:22.692 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:22.690+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 56.313004962000001, 2024-09-15T21:32:22.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:22.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:32:22.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 56.312091930000001, 2024-09-15T21:32:22.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:32:22.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 56.206223657999999, 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.767 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:22.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:32:22.771 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 56.205811388000001, 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.772 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:32:22.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.776 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 56.205363218999999, 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:22.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:32:22.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.781 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 56.204524870999997, 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.786 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 56.203608404999997, 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:32:22.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.789 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.791 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.753180835000002, 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:22.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:32:22.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.794 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:32:22.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.796 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.752734076000003, 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.797 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:32:22.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:32:22.800 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:32:22.801 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.752306834000002, 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.802 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.804 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:32:22.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:22.806 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.751839885000003, 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:22.808 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.808 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.809 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:22.809 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:32:22.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.809 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.809 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:32:22.809 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:32:22.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:32:22.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:22.812 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.751471342000002, 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.813 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:32:22.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:32:22.815 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.751054271999998, 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:32:22.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:32:22.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:32:22.821 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.748403256000003, 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.747705926999998, 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:32:22.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:32:22.829 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:32:22.830 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.464381033000002, 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.832 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.833 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.460391102999999, 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:32:22.835 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:32:22.837 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.838 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:32:22.839 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.459734841, 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.840 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:32:22.841 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.842 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:32:22.843 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.844 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.455281282999998, 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.845 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.846 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:32:22.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.848 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.849 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 55.452548528000001, 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:22.850 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.851 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.852 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.853 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.854 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:22.854 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:22.854 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:22.855 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:22.856 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":22,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:22.856 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:22.853+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:22.876 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:22.876 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:22.893 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:22.893 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:22.902 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.4 primary-affinity to 0.870335 (8570382) 2024-09-15T21:32:22.907 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:32:22.912 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:22.912 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:22.931 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:22.931 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:22.949 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:22.949 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:22.955 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:22.953+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:22.966 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:22.966 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:22.983 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:22.983 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:22.989 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:22.990 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:22.998 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:23.001 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:23.001 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:23.057 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:23.055+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:23.107 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:23.108 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:23.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:23.160 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:23.158+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:23.224 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:23.224 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:23.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:23.258 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:23.256+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:23.358 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:23.356+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:23.381 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:23.381 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:23.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:23.460 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:23.458+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:23.497 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:23.497 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:23.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:23.591 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:23.589+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:23.654 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:23.654 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:23.661 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:23.659+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:23.663 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:23.762 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:23.760+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:23.810 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:23.810 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:23.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:23.863 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:23.861+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:23.927 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:23.927 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:23.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:23.979 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:23.977+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:24.043 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:24.043 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:24.051 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:24.096 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:24.094+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:24.164 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:24.162+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:24.167 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:24.167 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:24.177 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:24.265 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:24.263+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:24.321 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:24.321 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:24.329 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:24.366 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:24.364+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:24.467 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:24.465+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:24.474 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:24.474 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:24.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:24.568 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:24.566+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:24.587 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:24.587 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:24.596 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:24.669 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:24.667+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:24.739 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:24.739 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:24.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:24.792 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:24.790+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:24.850 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:24.850 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:24.858 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:24.871 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:24.869+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:24.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:32:24.972 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:24.970+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:25.002 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:25.002 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:25.011 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:25.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:32:25.073 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:25.071+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:25.113 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:25.113 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:25.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:25.174 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:25.172+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:25.226 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:25.226 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:25.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:25.276 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:25.274+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:25.376 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:25.374+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:25.378 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:25.378 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:25.387 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:25.476 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:25.474+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:25.488 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:25.488 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:25.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:25.577 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:25.575+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:25.599 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:25.600 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:25.608 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:25.678 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:25.676+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:25.723 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:25.723 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:25.731 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:25.816 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:25.814+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:25.862 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:25.880 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:25.880 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:25.880 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:25.878+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:25.889 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:25.981 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:25.979+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:26.038 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:26.038 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:26.047 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:26.091 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:26.089+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:26.154 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:26.154 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:26.163 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:26.182 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:26.181+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:26.284 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:26.282+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:26.310 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:26.310 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:26.319 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:26.403 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:26.401+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:26.467 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:26.467 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:26.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:26.517 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:26.515+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:26.586 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:26.584+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:26.624 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:26.624 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:26.633 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:26.687 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:26.685+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:26.740 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:26.740 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:26.749 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:26.788 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:26.786+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:26.856 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:26.856 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:26.864 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:26.888 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:26.886+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:26.972 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:26.972 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:26.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:27.024 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:27.022+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:27.088 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:27.088 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:27.090 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:27.088+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:27.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:27.192 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:27.190+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:27.244 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:27.244 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:27.252 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:27.293 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:27.291+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:27.367 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:27.367 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:27.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:27.420 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:27.418+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:27.480 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:27.480 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:27.489 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:27.530 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:27.528+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:27.594 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:27.592+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:27.632 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:27.632 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:27.641 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:27.725 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:27.723+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:27.783 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:27.784 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:27.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:27.795 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:27.793+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:27.896 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:27.894+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:27.927 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:32:27.927 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:32:27.927 INFO:tasks.thrashosds.thrasher:Removing osd 7, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:32:27.927 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 7 2024-09-15T21:32:27.935 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:27.935 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:27.944 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:28.028 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:28.026+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:28.087 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:28.087 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:28.096 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:28.097 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:28.096+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:28.199 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:28.197+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:28.239 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:28.240 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:28.249 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:28.299 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:28.298+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:28.352 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:28.352 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:28.360 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:28.404 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:28.402+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:28.463 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:28.463 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:28.471 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:28.513 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:28.511+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:28.603 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:28.601+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:28.616 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:28.616 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:28.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:28.704 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:28.702+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:28.768 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:28.768 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:28.777 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:28.821 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:28.819+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:28.883 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:28.883 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:28.892 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:28.894 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:28.895 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:28.917 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:28.918 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:28.933 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:28.931+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:28.937 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:28.937 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:28.957 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:28.957 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:28.975 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:28.975 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:28.993 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:28.993 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:29.006 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:29.004+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:29.009 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:29.009 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:29.011 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:29.011 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:29.011 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:29.011 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:29.021 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:29.026 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:29.027 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:29.107 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:29.105+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:29.135 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:29.135 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:29.135 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:29.135 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:29.136 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:29.140 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":26,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:29.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:29.200 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-09-15T21:32:29.194+0000 7f0e898c4640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-15T21:32:29.208 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:29.207+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:29.254 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.810682399000001, 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.255 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:29.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:32:29.259 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.809769367000001, 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.261 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.703901094999999, 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:29.265 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.266 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:32:29.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.268 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.269 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.703488825000001, 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.271 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:32:29.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.273 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.274 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.703040655999999, 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.276 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.278 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:32:29.279 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.702202307999997, 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:32:29.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:32:29.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.701285841999997, 2024-09-15T21:32:29.284 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.285 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:32:29.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.287 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:32:29.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:29.289 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.250858272000002, 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.290 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:32:29.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.293 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.250411513000003, 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.294 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.295 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:32:29.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.297 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:32:29.298 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.249984271000002, 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.299 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:29.300 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.301 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:32:29.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:32:29.303 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:32:29.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.304 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.304 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.304 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:32:29.304 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.249517322000003, 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.305 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:32:29.307 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.308 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.249148779000002, 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.309 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:32:29.311 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.312 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:32:29.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.248731708999998, 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.314 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:32:29.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:32:29.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.246080693000003, 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:32:29.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.245383363999998, 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:32:29.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 61.962058470000002, 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:32:29.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 61.958068539999999, 2024-09-15T21:32:29.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:32:29.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 61.957412278, 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:32:29.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:32:29.342 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 61.952958719999998, 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:32:29.343 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:32:29.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 61.950225965000001, 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.348 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:32:29.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:32:29.352 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.353 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.353 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:29.354 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:29.355 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:29.352+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:29.455 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:29.453+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:29.503 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:29.503 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:29.503 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:29.503 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:29.512 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:29.596 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:29.594+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:29.656 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:29.654+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:29.659 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:29.660 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:29.660 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:29.660 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:29.660 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:29.669 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:29.757 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:29.755+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:29.816 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:29.816 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:29.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:29.816 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:29.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 57.357299228000002, 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:32:29.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:29.821 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 56.351417836000003, 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:29.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:32:29.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:29.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:32:29.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:32:29.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.826 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:29.826 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:29.826 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:29.827 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:29.855 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-09-15T21:32:29.853+0000 7f0e898c4640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-15T21:32:29.899 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:29.897+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:29.917 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:32:29.919 INFO:teuthology.orchestra.run.smithi019.stderr:marked out osd.7. 2024-09-15T21:32:29.966 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:29.967 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:29.967 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:29.967 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:29.967 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:29.965+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:29.975 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:30.060 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:30.058+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:30.122 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:30.122 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:30.122 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:30.122 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:30.122 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:30.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:30.172 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:30.170+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:30.262 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:30.260+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.278 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 58.821033643, 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:30.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:32:30.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:30.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:30.283 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:30.287 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:30.371 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:30.369+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:30.435 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:30.435 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:30.435 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:30.435 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:30.445 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:30.486 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:30.484+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:30.565 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:30.563+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:30.592 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:30.593 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:30.593 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:30.593 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:30.593 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:30.602 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:30.687 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:30.685+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:30.750 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:30.750 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:30.750 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:30.750 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [] 2024-09-15T21:32:30.750 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:30.759 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:30.803 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:30.801+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:30.868 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:30.866+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:30.875 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:30.875 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:30.875 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:30.875 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:30.884 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:30.969 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:30.967+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:31.030 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:31.030 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:31.030 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:31.030 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:31.030 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:31.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:32:31.042 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:31.086 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:31.084+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:31.146 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:31.146 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:31.147 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:31.156 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:31.168 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:31.168 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:31.171 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:31.169+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:31.185 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:31.185 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:31.201 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:31.201 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:31.217 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:31.217 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:31.234 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:31.234 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:31.251 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:31.251 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:31.268 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:31.268 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:31.271 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:31.269+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:31.286 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:31.286 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:31.302 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:31.302 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:31.311 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:31.395 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:31.393+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:31.454 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:31.454 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:31.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:31.473 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:31.471+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:31.574 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:31.572+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:31.604 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:31.604 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:31.613 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:31.675 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:31.673+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:31.717 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:31.717 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:31.726 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:31.775 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:31.773+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:31.829 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:31.829 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:31.837 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:31.877 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:31.875+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:31.933 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:32:31.940 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:31.940 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:31.949 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:31.993 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:31.991+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:32.052 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:32.052 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:32.061 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:32.078 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:32.077+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:32.156 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:32.179 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:32.177+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:32.204 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:32.205 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:32.213 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:32.297 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:32.295+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:32.356 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:32.365 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:32.406 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:32.404+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:32.473 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:32.473 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:32.482 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:32.480+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:32.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:32.531 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:32.545 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-09-15T21:32:32.545 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:32:32.545 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:32:32.558 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:32:32.558 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-09-15T21:32:32.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:32.594 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:32.594 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:32.600 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:32.598+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:32.603 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:32.683 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:32.681+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:32.753 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:32.763 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:32.783 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:32.782+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:32.871 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:32.871 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:32.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:32.924 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:32.922+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:32.932 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:32.987 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:32.985+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:32.988 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:32.988 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:32.997 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:33.086 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:33.084+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:33.145 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:33.145 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:33.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:33.195 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:33.193+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:33.288 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:33.286+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:33.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:32:33.302 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:33.303 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:33.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:33.389 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:33.387+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:33.426 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:33.426 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:33.443 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:33.444 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:33.460 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:33.460 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:33.461 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:33.461 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:33.461 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:33.461 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:33.461 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:33.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:33.476 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:33.477 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:33.491 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:33.489+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:33.494 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:33.494 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:33.511 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:33.511 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:33.528 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:33.528 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:33.545 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:33.545 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:33.579 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:33.579 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:33.579 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:33.579 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:33.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.121847185, 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:32:33.581 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:33.582 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:33.583 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:33.584 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:32:33.584 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:32:33.584 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:33.584 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:33.584 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:33.584 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:33.584 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:33.584 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:33.588 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:33.630 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:33.627+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:33.692 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:33.691+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:33.736 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:33.736 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:33.736 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:33.736 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:33.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:33.793 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:33.791+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:33.852 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:33.853 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:33.853 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:33.853 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:33.853 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:33.861 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:33.894 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:33.891+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:33.995 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:33.993+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:34.011 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:34.012 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:34.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:34.012 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [] 2024-09-15T21:32:34.012 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:34.021 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:34.096 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:34.094+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:34.136 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:34.136 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:34.136 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:34.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:34.197 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:34.195+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:34.249 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:34.249 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:34.249 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:34.249 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:34.249 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:34.257 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:34.299 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:34.297+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:34.399 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:34.397+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:34.401 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:34.401 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:34.401 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:34.409 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:34.500 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:34.498+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:34.513 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:34.513 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:34.513 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:34.513 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:34.521 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:34.601 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:34.599+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:34.624 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:34.625 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:34.625 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:34.625 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:34.625 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:34.633 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:34.702 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:34.700+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.282278857999998, 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:32:34.737 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:34.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:32:34.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:34.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.276139227999998, 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:32:34.744 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:32:34.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:34.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:34.748 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.272971052000003, 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:32:34.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 63.272425298999998, 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:32:34.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:34.754 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:32:34.756 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 62.279786991000002, 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:34.757 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:32:34.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:32:34.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:32:34.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:32:34.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 62.276494163000002, 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:32:34.764 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:32:34.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.273943393000003, 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:34.769 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:32:34.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:32:34.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:32:34.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.270542704, 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:34.776 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:32:34.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:34.780 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:34.781 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:34.803 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:34.801+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:34.884 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:34.884 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:34.884 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:34.884 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:34.893 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:34.903 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:34.901+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:34.935 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:32:34.935 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T21:32:34.995 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:34.995 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:34.995 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:34.995 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:34.995 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:35.004 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:35.004 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:35.002+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:35.105 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:35.103+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:35.147 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:35.147 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 12.233013154, 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:35.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:32:35.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2066757350000001, 2024-09-15T21:32:35.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:35.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 4.2032424669999999, 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:32:35.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:32:35.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:32:35.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.161 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.161 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.161 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:35.162 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:35.206 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:35.204+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:35.264 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:35.265 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:35.265 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:35.273 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:35.317 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:35.315+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:35.375 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:35.375 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:35.375 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:35.375 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:35.375 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:35.383 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:35.424 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:35.422+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:35.509 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:35.507+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.107958607, 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:32:35.529 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:32:35.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 69.084613761, 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:32:35.536 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:35.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:32:35.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.543 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.543 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.983246777999994, 2024-09-15T21:32:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:32:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.544 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:35.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:32:35.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:32:35.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.983062959999998, 2024-09-15T21:32:35.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:35.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:32:35.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:32:35.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.556 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.983024115000006, 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:32:35.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:32:35.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.982972141000005, 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.563 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:32:35.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:32:35.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.568 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.982956412999997, 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:32:35.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:32:35.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.982937765000003, 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:35.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:32:35.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.982879814, 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.581 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:32:35.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.982861983000006, 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:32:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:32:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.982780957000003, 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:32:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:32:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:32:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.982679701999999, 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:32:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:32:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.982633073000002, 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:32:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.525289220000005, 2024-09-15T21:32:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:32:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.52522003, 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:32:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.524954773999994, 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:32:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:32:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.524880503999995, 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:32:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.524851603000002, 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:32:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:32:35.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.524836274999998, 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:35.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:32:35.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.524751987000002, 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:35.650 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:32:35.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:35.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:35.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:35.656 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:35.656 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:32:35.658 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:35.656+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:35.658 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:35.759 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:35.757+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:35.770 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:35.770 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:35.770 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:35.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:35.785 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:35.786 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:35.803 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:35.803 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:35.819 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:35.820 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:35.836 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:35.836 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:35.853 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:35.853 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:35.860 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:35.859+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:35.870 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:35.870 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:35.887 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:35.887 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:35.894 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:35.894 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:35.903 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:35.904 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:35.905 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:35.946 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:35.961 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:35.960+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:36.012 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:36.012 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:36.021 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:36.065 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:36.063+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:36.132 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:36.132 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:36.141 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:36.182 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:36.180+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:36.264 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:36.262+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:36.289 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:36.289 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:36.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:36.331 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:36.385 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:36.383+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:36.406 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:36.406 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:36.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:36.499 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:36.497+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:36.564 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:36.564 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:36.566 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:36.564+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:36.571 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:36.667 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:36.666+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:36.718 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:36.718 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:36.727 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:36.769 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:36.767+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:36.835 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:36.836 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:36.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:36.888 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:36.886+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:36.952 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:36.952 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:36.960 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:37.002 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:37.000+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:37.072 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:37.070+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:37.108 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:37.108 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:37.117 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:37.173 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:37.172+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:37.264 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:37.264 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:37.272 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:37.274 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:37.272+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:37.375 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:37.373+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:37.388 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:37.388 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:37.397 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:37.476 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:37.474+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:37.501 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:37.501 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:37.509 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:37.577 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:37.575+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:37.613 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:37.613 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:37.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:37.705 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:37.703+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:37.764 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:37.764 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:37.772 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:37.779 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:37.777+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:37.880 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:37.878+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:37.914 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:37.915 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:37.917 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:32:37.923 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:37.981 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:37.979+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:38.025 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:38.025 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:38.034 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:38.082 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:38.080+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:38.137 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:38.138 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:38.146 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:38.187 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:38.185+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:38.284 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:38.282+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:38.289 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:38.289 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:38.298 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:38.385 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:38.382+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:38.443 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:38.443 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:38.451 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:38.493 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:38.491+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:38.587 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:38.585+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:38.595 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:38.595 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:38.604 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:38.687 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:38.686+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:38.707 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:38.707 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:38.716 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:38.789 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:38.787+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:38.819 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:38.820 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:38.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:38.890 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:38.888+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:38.950 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:38.950 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:38.959 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:39.001 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:38.999+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:39.092 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:39.091+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:39.108 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:39.109 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:39.118 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:39.193 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:39.191+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:39.225 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:39.225 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:39.233 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:39.318 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:39.316+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:39.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:39.382 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:39.383 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:39.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:39.433 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:39.431+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:39.495 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:39.494+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:39.542 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:39.542 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:39.551 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:39.635 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:39.633+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:39.697 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:39.696+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:39.699 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:39.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:39.718 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:39.798 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:39.796+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:39.826 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:39.826 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:39.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:39.900 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:39.898+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:39.941 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:39.941 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:39.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:40.001 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:39.999+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:40.058 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:40.058 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:40.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:40.102 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:40.100+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:40.202 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:40.201+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:40.213 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:40.213 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:40.222 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:40.304 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:40.302+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:40.329 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:40.329 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:40.338 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:40.404 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:40.402+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:40.445 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:40.445 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:40.453 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:40.506 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:40.504+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:40.568 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:40.568 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:40.578 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:40.622 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:40.620+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:40.681 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:40.681 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:40.690 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:40.731 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:40.729+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:40.808 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:40.806+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:40.833 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:40.833 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:40.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:40.908 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:40.906+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:40.944 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:40.944 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:40.954 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:41.038 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:41.036+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:41.098 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:41.098 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:41.107 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:41.148 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:41.146+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:41.211 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:41.209+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:41.250 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:41.251 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:41.259 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:41.312 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:41.310+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:41.403 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:41.404 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:41.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:41.413 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:41.411+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:41.514 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:41.514 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:41.515 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:41.512+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:41.522 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:41.615 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:41.613+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:41.624 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:41.625 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:41.633 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:41.716 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:41.714+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:41.736 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:41.736 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:41.745 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:41.829 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:41.827+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:41.888 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:41.888 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:41.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:41.938 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:41.937+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:42.018 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:42.016+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:42.041 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:42.041 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:42.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:42.119 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:42.118+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:42.171 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:42.171 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:42.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:42.220 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:42.219+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:42.289 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:42.289 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:42.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:42.343 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:42.341+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:42.407 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:42.407 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:42.416 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:42.457 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:42.455+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:42.522 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:42.521+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:42.563 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:42.564 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:42.572 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:42.623 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:42.621+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:42.679 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:42.680 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:42.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:42.723 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:42.722+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:42.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:42.797 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:42.797 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:42.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:42.851 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:42.849+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:42.915 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:42.915 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:42.924 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:42.925 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:42.924+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:43.027 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:43.025+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:43.072 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:43.072 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:43.080 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:43.127 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:43.126+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:43.187 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:43.187 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:43.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:43.228 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:43.226+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:43.303 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:43.303 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:43.312 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:43.356 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:43.354+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:43.419 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:43.419 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:43.428 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:43.430 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:43.429+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:43.531 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:43.529+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:43.575 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:43.576 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:43.584 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:43.632 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:43.630+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:43.699 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:43.699 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:43.708 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:43.734 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:43.732+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:43.811 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:43.811 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:43.821 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:43.835 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:43.833+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:43.924 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:43.924 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:43.932 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:43.937 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:43.935+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:44.034 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:44.034 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:44.037 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:44.036+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:44.043 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:44.139 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:44.137+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:44.185 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:44.185 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:44.194 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:44.239 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:44.238+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:44.297 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:44.297 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:44.306 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:44.341 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:44.339+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:44.441 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:44.440+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:44.450 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:44.450 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:44.458 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:44.542 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:44.540+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:44.602 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:44.602 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:44.610 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:44.655 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:44.654+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:44.713 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:44.721 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:44.744 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:44.742+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:44.845 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:44.843+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:44.863 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:44.873 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:44.945 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:44.943+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:44.975 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:44.975 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:44.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:45.046 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:45.044+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:45.085 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:45.085 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:45.095 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:45.147 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:45.145+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:45.216 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:45.217 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:45.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:45.247 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:45.246+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:45.349 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:45.348+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:45.375 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:45.375 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:45.384 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:45.450 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:45.448+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:45.491 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:45.491 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:45.500 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:45.551 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:45.548+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:45.607 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:45.607 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:45.615 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:45.651 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:45.649+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:45.722 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:45.722 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:45.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:45.776 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:45.774+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:45.839 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:45.839 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:45.847 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:45.889 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:45.887+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:45.954 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:45.952+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:45.995 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:45.996 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:46.004 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:46.055 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:46.053+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:46.111 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:46.111 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:46.120 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:46.155 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:46.153+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:46.227 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:46.228 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:46.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:46.280 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:46.278+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:46.343 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:46.344 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:46.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:46.393 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:46.391+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:46.459 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:46.457+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:46.499 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:46.500 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:46.508 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:46.593 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:46.591+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:46.657 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:46.657 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:46.661 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:46.659+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:46.665 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:46.762 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:46.760+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:46.781 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:46.781 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:46.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:46.863 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:46.861+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:46.893 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:46.894 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:46.902 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:46.964 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:46.962+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:47.004 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:47.004 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:47.014 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:47.065 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:47.063+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:47.116 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:47.116 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:47.124 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:47.163 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:47.163 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:47.165 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:47.164+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:47.181 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:47.181 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:47.198 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:47.199 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:47.216 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:47.216 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:47.226 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:47.226 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:47.232 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:47.233 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:47.236 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:47.250 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:47.250 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:47.267 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:47.265+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:47.267 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:47.267 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:47.284 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:47.284 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:47.340 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 75.885950605999994, 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:47.341 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:32:47.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:32:47.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:47.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:32:47.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 75.879810976000002, 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:32:47.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:32:47.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:32:47.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.353 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 75.876642799999999, 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:32:47.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:32:47.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:32:47.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:32:47.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:32:47.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:32:47.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:32:47.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:32:47.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.359 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.360 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.360 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:47.360 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 75.876097047000002, 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:32:47.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.362 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:47.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:32:47.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.883458739000005, 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:47.366 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:32:47.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:32:47.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:32:47.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 74.880165911000006, 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:47.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:32:47.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.877615141000007, 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.378 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:32:47.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:32:47.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:32:47.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.874214452000004, 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:32:47.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.389 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:47.391 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:47.391 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:47.388+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:47.492 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:47.490+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:47.534 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:47.535 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:47.535 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:47.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:47.593 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:47.591+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:47.647 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:47.647 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:47.647 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:47.647 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:47.647 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:47.656 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:47.699 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:47.697+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:47.758 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:47.758 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:47.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.844041558000001, 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.759 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:32:47.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.763 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.817704139, 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:32:47.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:32:47.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.814270871000002, 2024-09-15T21:32:47.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:47.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:32:47.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.773 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:47.773 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:47.773 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:47.774 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:47.794 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:47.792+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:47.895 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:47.893+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:47.917 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:47.917 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:47.918 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:47.926 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T21:32:47.945 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:32:47.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T21:32:48.005 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:48.003+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:48.069 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:48.069 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:48.069 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:48.069 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:48.069 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:48.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:48.121 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:48.119+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:48.182 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:48.182 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:48.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:48.182 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:48.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.182 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:32:48.182 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 82.761219982, 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:32:48.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:32:48.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:32:48.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.189 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.737875136, 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.190 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:48.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:32:48.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:32:48.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636508152999994, 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:32:48.196 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:32:48.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636324334999998, 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.203 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:32:48.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636285490000006, 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.209 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:32:48.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:32:48.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.215 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:48.216 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636233516000004, 2024-09-15T21:32:48.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:32:48.216 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.216 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.216 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.216 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:48.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:32:48.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:32:48.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636217787999996, 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:32:48.222 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:32:48.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.228 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636199140000002, 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:32:48.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636141189, 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.235 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:32:48.238 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:32:48.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636123358000006, 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.241 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:32:48.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:32:48.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.245 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:32:48.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.636042332000002, 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.248 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:32:48.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:32:48.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:32:48.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:32:48.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:32:48.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.635941076999998, 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:32:48.255 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:32:48.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.635894448000002, 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:32:48.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:32:48.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.267 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.178550595000004, 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.268 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:32:48.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:32:48.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.178481404999999, 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.274 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:48.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:32:48.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:32:48.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.178216148999994, 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:32:48.280 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:32:48.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.178141878999995, 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.287 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:32:48.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.292 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.178112978000001, 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.293 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:32:48.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:32:48.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.178097649999998, 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.299 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:48.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:32:48.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:32:48.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.178013362000002, 2024-09-15T21:32:48.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:48.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:32:48.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:32:48.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:48.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:48.311 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:48.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:48.313 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:48.311+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:48.413 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:48.411+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:48.421 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:48.421 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:48.421 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:48.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:48.514 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:48.512+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:48.536 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:48.536 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:48.537 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:48.537 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:48.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:48.630 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:48.628+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:48.695 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:48.695 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:48.695 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:48.695 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:48.695 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.695 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 82.251226880000004, 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:48.696 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.697 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:32:48.698 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.699 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.700 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 82.250313848000005, 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.702 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:32:48.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.704 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:32:48.705 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 82.144445575999995, 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.706 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.707 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:32:48.708 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.709 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 82.144033305999997, 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:32:48.710 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.712 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.713 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:32:48.714 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 82.143585137000002, 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.715 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.716 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:32:48.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:32:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:32:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.718 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.719 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 82.142746789, 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:32:48.720 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:48.721 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.722 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:32:48.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.724 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 82.141830322999994, 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.725 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.727 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.728 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.729 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.691402753000006, 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.730 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.731 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:32:48.732 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:32:48.734 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.690955994000007, 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:32:48.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.690528752000006, 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.740 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:48.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:32:48.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.690061803000006, 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.746 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.746 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.746 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.746 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:48.746 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:32:48.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.689693259999999, 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:32:48.750 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.752 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:32:48.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.754 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.689276190000001, 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.755 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.756 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:32:48.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:32:48.759 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.686625174, 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:32:48.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:32:48.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.764 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.685927844999995, 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:32:48.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.402602951000006, 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:32:48.769 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:32:48.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.773 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.398613021000003, 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:32:48.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.397956758999996, 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:32:48.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:32:48.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.393503201000001, 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:32:48.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.390770446000005, 2024-09-15T21:32:48.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:48.789 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:48.790 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:32:48.792 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.793 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.793 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.793 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:48.793 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:48.793 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:48.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:48.795 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:48.792+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:48.895 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:48.893+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:48.944 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:48.945 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:48.945 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:48.945 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:48.945 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T21:32:48.957 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:48.962 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T21:32:48.996 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:48.994+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:49.096 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:49.094+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:49.110 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":32,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:49.112 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:49.112 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:49.112 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:49.112 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:49.112 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:49.123 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:49.126 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-09-15T21:32:49.126 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:32:49.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:32:49.129 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:32:49.129 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-09-15T21:32:49.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:49.210 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:49.208+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:49.236 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 76.777694582999999, 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:49.237 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:49.238 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.239 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:49.240 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:32:49.240 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:32:49.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:49.242 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:32:49.242 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:32:49.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:32:49.243 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 75.771813191000007, 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.244 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:32:49.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:32:49.246 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:49.247 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:32:49.248 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:32:49.248 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:49.248 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:49.248 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:49.248 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:49.248 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:49.248 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:49.249 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:49.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:32:49.334 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:49.332+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:49.399 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:49.397+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:49.400 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:49.400 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:49.400 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:49.400 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:49.410 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:49.479 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:49.480 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:49.497 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:49.497 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:49.500 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:49.498+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:49.515 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:49.515 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:49.522 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:49.522 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:49.532 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:49.533 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:49.533 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:49.552 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:49.552 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:49.565 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":32,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:49.570 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:49.570 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:49.588 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:49.588 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:49.601 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:49.599+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:49.605 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:49.605 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:49.651 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:49.652 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:49.661 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:49.702 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:49.700+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:49.803 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:49.801+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:49.809 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:49.809 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:49.818 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:49.904 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:49.902+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:49.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:32:49.967 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:49.967 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:49.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:50.019 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:50.017+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:50.106 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:50.104+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:50.129 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:50.129 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:50.139 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:50.193 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T21:32:50.207 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T21:32:50.208 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:50.205+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:50.251 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:50.251 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:50.260 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:50.344 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:50.342+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:50.404 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:50.404 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:50.408 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:50.406+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:50.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:50.509 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:50.507+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:50.557 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:50.557 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:50.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:50.610 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:50.608+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:50.711 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:50.711 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:50.711 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:50.709+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:50.719 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:50.812 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:50.810+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:50.862 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:50.862 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:50.870 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:50.915 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:50.913+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:50.974 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:50.974 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:50.982 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:51.023 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:51.021+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:51.117 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:51.115+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:51.126 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:51.126 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:51.135 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:32:51.194 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T21:32:51.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T21:32:51.217 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:51.215+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:51.238 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:51.238 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:51.246 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:51.331 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:51.329+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:51.391 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:51.391 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:51.401 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:51.419 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:51.417+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:51.522 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:51.519+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:51.544 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:51.544 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:51.552 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:51.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:32:51.622 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:51.620+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:51.655 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:51.655 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:51.663 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:51.745 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:51.746 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:51.747 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:51.745+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:51.762 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:51.762 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:51.778 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:51.779 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:51.794 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:51.795 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:51.805 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:51.806 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:51.811 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:51.811 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:51.814 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:51.828 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:51.828 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:51.845 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:51.846 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:51.855 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:51.853+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:51.862 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:51.862 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:51.923 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:51.924 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:51.924 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:51.924 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:51.925 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:51.923+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:51.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:52.026 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:52.024+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:52.040 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:52.040 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:52.040 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:52.040 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:52.040 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:52.049 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:52.134 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:52.132+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T21:32:52.200 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:52.200 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:52.200 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:52.200 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:52.200 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.200 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:32:52.200 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:52.200 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.756284199000007, 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.201 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:32:52.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.204 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.755371167000007, 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:32:52.205 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.207 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.209 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.649502894999998, 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.210 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.211 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.212 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:32:52.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.649090624999999, 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:52.215 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:32:52.218 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.648642456000005, 2024-09-15T21:32:52.219 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:32:52.220 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.220 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.220 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.220 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.220 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:32:52.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:32:52.224 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.647804108000003, 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.225 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:32:52.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.227 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.228 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.646887641999996, 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:32:52.229 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.231 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.232 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:32:52.233 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.196460071999994, 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.234 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:32:52.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.236 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.237 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:32:52.238 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.196013312999995, 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.239 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:32:52.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.241 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.242 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.195586070999994, 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.244 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:32:52.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.246 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.247 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.195119121999994, 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.248 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:32:52.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:32:52.250 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.251 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.194750579000001, 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:32:52.254 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.255 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.194333509000003, 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:32:52.257 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:32:52.259 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.260 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:32:52.261 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.191682493000002, 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:32:52.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.264 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.265 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.190985163999997, 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:32:52.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.269 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.270 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 84.907660269999994, 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.271 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:32:52.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.274 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.275 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 84.903670340000005, 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.276 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:32:52.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:32:52.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:32:52.280 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 84.903014077999998, 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.282 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:32:52.283 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:32:52.284 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 84.898560520000004, 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.285 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.286 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.287 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:32:52.288 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.289 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 84.895827765000007, 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:52.290 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.291 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:32:52.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.294 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:52.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:52.296 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T21:32:52.297 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:52.293+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:52.396 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:52.394+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:52.448 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:52.449 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:52.449 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:52.449 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:52.457 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:52.497 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:52.495+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:52.586 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:52.605 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:52.605 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:52.605 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:52.605 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:52.605 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:52.606 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:52.603+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:52.615 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:52.700 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:52.698+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 80.306039154999993, 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:32:52.765 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:32:52.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 79.300157763000001, 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:52.770 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:32:52.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:32:52.774 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.775 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:52.775 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:52.775 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:52.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:52.821 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:52.819+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:52.884 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:52.885 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:52.885 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:52.885 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:52.893 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:52.935 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:52.933+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:52.967 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":32,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:53.004 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:53.001+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:53.043 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:53.043 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:53.043 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:53.043 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:53.043 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:53.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:53.104 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:53.102+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:53.160 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.701271269000003, 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:53.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:53.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:53.162 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:53.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:53.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:53.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:53.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.164 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:32:53.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:53.167 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:53.168 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:53.169 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:53.199 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T21:32:53.205 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:53.203+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:53.214 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:32:53.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T21:32:53.281 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:53.281 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:53.281 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:53.281 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:53.290 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:53.306 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:53.304+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:53.408 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:53.406+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:53.441 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:53.441 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:53.442 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:53.442 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:53.442 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:53.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:53.509 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:53.507+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:53.559 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:53.559 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:53.559 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:53.559 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [] 2024-09-15T21:32:53.559 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:53.569 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:53.610 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:53.608+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:53.684 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:53.685 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:53.685 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:53.685 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:53.694 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:53.710 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:53.708+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:53.798 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:53.799 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:53.799 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:53.807 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:53.852 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:53.850+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:53.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:32:53.909 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:53.909 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:53.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:53.909 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:53.909 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:53.911 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:53.909+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:53.918 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:54.002 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:54.002 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:54.012 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:54.010+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:54.019 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:54.020 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:54.036 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:54.036 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:54.052 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:54.052 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:54.061 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:54.061 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:54.061 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:54.061 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:54.068 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:54.068 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:54.071 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:54.085 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:54.086 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:54.102 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:54.102 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:54.113 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:54.111+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:54.118 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:54.118 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:54.174 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:54.174 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:54.182 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:54.217 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T21:32:54.225 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:32:54.226 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:54.224+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:54.235 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T21:32:54.236 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:54.287 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:54.287 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:54.296 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:54.338 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:54.335+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:54.417 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:54.415+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:54.440 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:54.441 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:54.448 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:54.518 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:54.516+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:54.533 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:32:54.533 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:32:54.545 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2024-09-15T21:32:54.222181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":191592,"kb_used_data":2364,"kb_used_omap":10,"kb_used_meta":189045,"kb_avail":660411288,"statfs":{"total":676457349120,"available":676261158912,"internally_reserved":0,"allocated":2420736,"data_stored":1946106,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":11162,"internal_metadata":193582182},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.031929"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:32:52.459958+0000","last_change":"2024-09-15T21:32:31.148685+0000","last_active":"2024-09-15T21:32:52.459958+0000","last_peered":"2024-09-15T21:32:52.459958+0000","last_clean":"2024-09-15T21:32:52.459958+0000","last_became_active":"2024-09-15T21:32:31.148363+0000","last_became_peered":"2024-09-15T21:32:31.148363+0000","last_unstale":"2024-09-15T21:32:52.459958+0000","last_undegraded":"2024-09-15T21:32:52.459958+0000","last_fullsized":"2024-09-15T21:32:52.459958+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:47.093586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.512702+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:32:50.512702+0000","last_peered":"2024-09-15T21:32:50.512702+0000","last_clean":"2024-09-15T21:32:50.512702+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:32:50.512702+0000","last_undegraded":"2024-09-15T21:32:50.512702+0000","last_fullsized":"2024-09-15T21:32:50.512702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.774058+0000","last_change":"2024-09-15T21:32:31.148878+0000","last_active":"2024-09-15T21:32:49.774058+0000","last_peered":"2024-09-15T21:32:49.774058+0000","last_clean":"2024-09-15T21:32:49.774058+0000","last_became_active":"2024-09-15T21:32:31.148553+0000","last_became_peered":"2024-09-15T21:32:31.148553+0000","last_unstale":"2024-09-15T21:32:49.774058+0000","last_undegraded":"2024-09-15T21:32:49.774058+0000","last_fullsized":"2024-09-15T21:32:49.774058+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:36.125113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.512704+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:32:50.512704+0000","last_peered":"2024-09-15T21:32:50.512704+0000","last_clean":"2024-09-15T21:32:50.512704+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:32:50.512704+0000","last_undegraded":"2024-09-15T21:32:50.512704+0000","last_fullsized":"2024-09-15T21:32:50.512704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:32:52.459951+0000","last_change":"2024-09-15T21:32:22.915752+0000","last_active":"2024-09-15T21:32:52.459951+0000","last_peered":"2024-09-15T21:32:52.459951+0000","last_clean":"2024-09-15T21:32:52.459951+0000","last_became_active":"2024-09-15T21:32:22.911196+0000","last_became_peered":"2024-09-15T21:32:22.911196+0000","last_unstale":"2024-09-15T21:32:52.459951+0000","last_undegraded":"2024-09-15T21:32:52.459951+0000","last_fullsized":"2024-09-15T21:32:52.459951+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:26.567449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:32:52.195725+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:32:52.195725+0000","last_peered":"2024-09-15T21:32:52.195725+0000","last_clean":"2024-09-15T21:32:52.195725+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:32:52.195725+0000","last_undegraded":"2024-09-15T21:32:52.195725+0000","last_fullsized":"2024-09-15T21:32:52.195725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.512702+0000","last_change":"2024-09-15T21:32:30.937398+0000","last_active":"2024-09-15T21:32:50.512702+0000","last_peered":"2024-09-15T21:32:50.512702+0000","last_clean":"2024-09-15T21:32:50.512702+0000","last_became_active":"2024-09-15T21:32:30.937117+0000","last_became_peered":"2024-09-15T21:32:30.937117+0000","last_unstale":"2024-09-15T21:32:50.512702+0000","last_undegraded":"2024-09-15T21:32:50.512702+0000","last_fullsized":"2024-09-15T21:32:50.512702+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:08:47.735206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":45,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:32:53.205468+0000","last_change":"2024-09-15T21:31:33.463323+0000","last_active":"2024-09-15T21:32:53.205468+0000","last_peered":"2024-09-15T21:32:53.205468+0000","last_clean":"2024-09-15T21:32:53.205468+0000","last_became_active":"2024-09-15T21:31:31.455121+0000","last_became_peered":"2024-09-15T21:31:31.455121+0000","last_unstale":"2024-09-15T21:32:53.205468+0000","last_undegraded":"2024-09-15T21:32:53.205468+0000","last_fullsized":"2024-09-15T21:32:53.205468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:01.950055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055004999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":120,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.774064+0000","last_change":"2024-09-15T21:32:30.945654+0000","last_active":"2024-09-15T21:32:49.774064+0000","last_peered":"2024-09-15T21:32:49.774064+0000","last_clean":"2024-09-15T21:32:49.774064+0000","last_became_active":"2024-09-15T21:32:30.936532+0000","last_became_peered":"2024-09-15T21:32:30.936532+0000","last_unstale":"2024-09-15T21:32:49.774064+0000","last_undegraded":"2024-09-15T21:32:49.774064+0000","last_fullsized":"2024-09-15T21:32:49.774064+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:47.118027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1586784,"data_compressed":17364,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":68719476757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476758,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":315392,"data_stored":119843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476758,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":315392,"data_stored":119843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476758,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":648771,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":368640,"data_stored":141778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":654269,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":339968,"data_stored":130801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":130801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:54.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:54.592 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:54.592 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:54.601 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:54.646 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:54.644+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:54.705 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:54.705 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:54.713 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:54.754 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:54.752+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:54.821 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:54.819+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:54.855 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:54.855 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:54.864 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:54.882 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:32:54.883 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:32:54.895 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-15T21:32:54.623004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":528947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":191592,"kb_used_data":2364,"kb_used_omap":10,"kb_used_meta":189045,"kb_avail":660411288,"statfs":{"total":676457349120,"available":676261158912,"internally_reserved":0,"allocated":2420736,"data_stored":1946106,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":11162,"internal_metadata":193582182},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000500"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":47,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:32:52.459958+0000","last_change":"2024-09-15T21:32:31.148685+0000","last_active":"2024-09-15T21:32:52.459958+0000","last_peered":"2024-09-15T21:32:52.459958+0000","last_clean":"2024-09-15T21:32:52.459958+0000","last_became_active":"2024-09-15T21:32:31.148363+0000","last_became_peered":"2024-09-15T21:32:31.148363+0000","last_unstale":"2024-09-15T21:32:52.459958+0000","last_undegraded":"2024-09-15T21:32:52.459958+0000","last_fullsized":"2024-09-15T21:32:52.459958+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:59:47.093586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.512702+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:32:50.512702+0000","last_peered":"2024-09-15T21:32:50.512702+0000","last_clean":"2024-09-15T21:32:50.512702+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:32:50.512702+0000","last_undegraded":"2024-09-15T21:32:50.512702+0000","last_fullsized":"2024-09-15T21:32:50.512702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.774058+0000","last_change":"2024-09-15T21:32:31.148878+0000","last_active":"2024-09-15T21:32:49.774058+0000","last_peered":"2024-09-15T21:32:49.774058+0000","last_clean":"2024-09-15T21:32:49.774058+0000","last_became_active":"2024-09-15T21:32:31.148553+0000","last_became_peered":"2024-09-15T21:32:31.148553+0000","last_unstale":"2024-09-15T21:32:49.774058+0000","last_undegraded":"2024-09-15T21:32:49.774058+0000","last_fullsized":"2024-09-15T21:32:49.774058+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:55:36.125113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.512704+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:32:50.512704+0000","last_peered":"2024-09-15T21:32:50.512704+0000","last_clean":"2024-09-15T21:32:50.512704+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:32:50.512704+0000","last_undegraded":"2024-09-15T21:32:50.512704+0000","last_fullsized":"2024-09-15T21:32:50.512704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":28,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:32:52.459951+0000","last_change":"2024-09-15T21:32:22.915752+0000","last_active":"2024-09-15T21:32:52.459951+0000","last_peered":"2024-09-15T21:32:52.459951+0000","last_clean":"2024-09-15T21:32:52.459951+0000","last_became_active":"2024-09-15T21:32:22.911196+0000","last_became_peered":"2024-09-15T21:32:22.911196+0000","last_unstale":"2024-09-15T21:32:52.459951+0000","last_undegraded":"2024-09-15T21:32:52.459951+0000","last_fullsized":"2024-09-15T21:32:52.459951+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:26.567449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":41,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-15T21:32:52.195725+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:32:52.195725+0000","last_peered":"2024-09-15T21:32:52.195725+0000","last_clean":"2024-09-15T21:32:52.195725+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:32:52.195725+0000","last_undegraded":"2024-09-15T21:32:52.195725+0000","last_fullsized":"2024-09-15T21:32:52.195725+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.512702+0000","last_change":"2024-09-15T21:32:30.937398+0000","last_active":"2024-09-15T21:32:50.512702+0000","last_peered":"2024-09-15T21:32:50.512702+0000","last_clean":"2024-09-15T21:32:50.512702+0000","last_became_active":"2024-09-15T21:32:30.937117+0000","last_became_peered":"2024-09-15T21:32:30.937117+0000","last_unstale":"2024-09-15T21:32:50.512702+0000","last_undegraded":"2024-09-15T21:32:50.512702+0000","last_fullsized":"2024-09-15T21:32:50.512702+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:08:47.735206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":45,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-15T21:32:53.205468+0000","last_change":"2024-09-15T21:31:33.463323+0000","last_active":"2024-09-15T21:32:53.205468+0000","last_peered":"2024-09-15T21:32:53.205468+0000","last_clean":"2024-09-15T21:32:53.205468+0000","last_became_active":"2024-09-15T21:31:31.455121+0000","last_became_peered":"2024-09-15T21:31:31.455121+0000","last_unstale":"2024-09-15T21:32:53.205468+0000","last_undegraded":"2024-09-15T21:32:53.205468+0000","last_fullsized":"2024-09-15T21:32:53.205468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:20:01.950055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055004999999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":120,"reported_epoch":30,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.774064+0000","last_change":"2024-09-15T21:32:30.945654+0000","last_active":"2024-09-15T21:32:49.774064+0000","last_peered":"2024-09-15T21:32:49.774064+0000","last_clean":"2024-09-15T21:32:49.774064+0000","last_became_active":"2024-09-15T21:32:30.936532+0000","last_became_peered":"2024-09-15T21:32:30.936532+0000","last_unstale":"2024-09-15T21:32:49.774064+0000","last_undegraded":"2024-09-15T21:32:49.774064+0000","last_fullsized":"2024-09-15T21:32:49.774064+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:17:47.118027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1586784,"data_compressed":17364,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":0,"seq":68719476757,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476758,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":315392,"data_stored":119843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476758,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":308,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":315392,"data_stored":119843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476758,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27388,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344452,"statfs":{"total":96636764160,"available":96608718848,"internally_reserved":0,"allocated":368640,"data_stored":648771,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542166,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":368640,"data_stored":141778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27396,"kb_used_data":364,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344444,"statfs":{"total":96636764160,"available":96608710656,"internally_reserved":0,"allocated":372736,"data_stored":654269,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":339968,"data_stored":130801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":339968,"data_stored":130801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:54.896 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T21:32:54.896 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:32:54.896 INFO:tasks.thrashosds.thrasher:Removing osd 5, in_osds are: [0, 1, 2, 3, 4, 5, 6] 2024-09-15T21:32:54.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 5 2024-09-15T21:32:54.941 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:54.939+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:55.008 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:55.008 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:55.017 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:55.023 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:55.021+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:55.120 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:55.120 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:55.124 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:55.122+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:55.129 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:55.208 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-09-15T21:32:55.203+0000 7f0e898c4640 -1 mon.a@0(leader).osd e35 definitely_dead 0 2024-09-15T21:32:55.225 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:55.223+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:55.228 INFO:teuthology.orchestra.run.smithi019.stderr:marked out osd.5. 2024-09-15T21:32:55.241 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:55.241 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:55.250 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:55.327 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:55.325+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:55.397 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:55.397 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:55.406 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:55.428 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:55.426+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:55.513 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:55.513 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:55.522 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:55.565 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:55.563+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:55.629 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:55.629 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:55.630 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:55.628+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:55.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:55.730 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:55.728+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:55.784 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:55.784 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:55.793 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:55.834 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:55.832+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:55.933 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:55.931+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:55.940 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:55.940 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:55.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:55.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:56.036 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:56.034+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:56.058 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:56.058 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:56.067 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:56.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:32:56.152 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:56.150+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:56.215 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:56.215 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:56.225 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:56.238 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:56.235+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:56.239 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:32:56.281 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:56.282 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:56.300 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:56.301 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:56.319 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:56.319 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:56.338 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:56.336+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:56.339 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:56.339 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:56.342 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:32:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:32:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 84.884722104999994, 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:32:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:32:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:32:56.347 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:56.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:56.358 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:56.358 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:56.374 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":32,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:56.376 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:56.376 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:56.401 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:56.401 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:56.418 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:56.419 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:56.470 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:56.471 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:56.471 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:56.471 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:56.469+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:56.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:56.564 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:56.562+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:56.627 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:56.628 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:56.628 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:32:56.628 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:32:56.628 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:56.637 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:32:56.641 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:56.639+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:56.742 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:56.740+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:56.784 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:56.784 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:32:56.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:32:56.784 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [] 2024-09-15T21:32:56.785 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:56.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:32:56.843 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:56.841+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:56.907 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:56.908 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:56.908 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:56.908 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:56.916 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:32:56.958 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:56.956+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:57.045 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:57.043+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:57.061 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:57.061 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:57.061 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:57.061 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:57.061 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:57.071 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:32:57.146 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:57.144+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:57.174 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:57.174 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:57.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:57.174 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:32:57.175 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:57.183 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:32:57.236 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:32:57.247 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:57.245+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:57.326 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:57.326 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:57.326 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:57.327 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:57.335 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:32:57.348 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:57.346+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:57.438 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:57.438 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:57.438 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:57.447 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:32:57.449 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:57.447+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:57.551 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:57.551 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:57.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:57.551 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:57.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.551 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:57.551 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.096238279000005, 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:32:57.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:32:57.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:32:57.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.090098648999998, 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:32:57.562 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.086930472999995, 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:57.563 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:32:57.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.086384719999998, 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:57.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:57.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:32:57.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:32:57.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:32:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:32:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:32:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:32:57.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:32:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.093746412000002, 2024-09-15T21:32:57.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.575 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:57.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:32:57.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:57.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:32:57.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:32:57.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:57.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:32:57.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:32:57.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:57.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:32:57.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:32:57.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:57.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:32:57.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:32:57.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:32:57.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:32:57.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:57.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:32:57.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:32:57.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:32:57.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:32:57.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:57.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:32:57.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:32:57.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:57.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:32:57.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:32:57.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:57.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:32:57.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:32:57.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:57.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:32:57.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:32:57.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:57.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:32:57.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:32:57.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:57.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:32:57.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:32:57.587 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.588 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.588 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.588 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.090453584000002, 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:57.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:32:57.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.087902814000003, 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:57.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:32:57.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:32:57.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:32:57.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.084502125, 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:57.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:57.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:57.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:32:57.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:32:57.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.604 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:57.604 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:57.604 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:57.604 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:57.549+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:57.605 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:32:57.689 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:57.687+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:57.749 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:57.750 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:57.750 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:57.750 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:57.753 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:57.751+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:57.758 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:32:57.854 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:32:57.852+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:32:57.902 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:57.902 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:57.902 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:57.902 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:57.902 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:57.910 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:32:57.955 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:57.953+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:58.054 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:58.054 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:58.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:58.054 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:58.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.054 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 35.139772587000003, 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:32:58.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.113435167999999, 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.059 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:32:58.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:32:58.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:58.063 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.1100019, 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:32:58.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.068 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.068 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:58.069 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:32:58.069 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:58.067+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:58.170 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:58.168+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:58.171 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:58.171 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:32:58.172 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:58.180 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:32:58.271 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:58.269+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:58.285 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:58.285 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:32:58.285 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:32:58.285 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:32:58.285 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:58.293 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:32:58.372 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:32:58.370+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:32:58.399 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:32:58.399 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:32:58.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 92.978067233000004, 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:32:58.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:32:58.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:32:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:32:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:32:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:32:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:32:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:32:58.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:32:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:32:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:32:58.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:32:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:32:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:32:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:32:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:32:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:32:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:32:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:32:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:32:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:32:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:32:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:32:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.954722387000004, 2024-09-15T21:32:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:32:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:32:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.414 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:32:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:32:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:32:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:32:58.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:32:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:32:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:32:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:32:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:32:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:32:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:32:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:32:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:32:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:32:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:32:58.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:32:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:32:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:32:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:32:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:32:58.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:32:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:32:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:32:58.424 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:58.425 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.853355403999998, 2024-09-15T21:32:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:32:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.426 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:32:58.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:32:58.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:32:58.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:32:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:32:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:32:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:32:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:32:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:32:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:32:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:32:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:32:58.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:32:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:32:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:32:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:32:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:32:58.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:32:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:32:58.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:32:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:32:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:32:58.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:32:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:32:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.438 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:58.439 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.853171586000002, 2024-09-15T21:32:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:32:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.440 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:32:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:32:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:32:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:32:58.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:32:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:32:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:32:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:32:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:32:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:32:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:32:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:32:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:32:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:32:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:32:58.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.853132740999996, 2024-09-15T21:32:58.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:32:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:32:58.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:32:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:32:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:32:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:32:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:32:58.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:32:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:32:58.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:32:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:32:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:32:58.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.853080766999994, 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:32:58.460 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:32:58.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:32:58.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.853065039000001, 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.466 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:32:58.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:32:58.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.853046391000007, 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.472 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:32:58.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:32:58.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.852988440000004, 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:32:58.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:32:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:32:58.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.852970608999996, 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:32:58.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:32:58.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:32:58.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:32:58.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:32:58.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.491 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.852889583000007, 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.492 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:32:58.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.852788328000003, 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.498 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:32:58.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:32:58.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.852741699000006, 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:32:58.504 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:32:58.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:32:58.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.395397845999995, 2024-09-15T21:32:58.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:32:58.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.395328656000004, 2024-09-15T21:32:58.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:32:58.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:32:58.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.395063399999998, 2024-09-15T21:32:58.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:32:58.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.394989129999999, 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:32:58.528 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:32:58.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.394960229000006, 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:32:58.534 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.394944901000002, 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.540 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:32:58.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:32:58.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 91.394860613000006, 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:32:58.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:32:58.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:32:58.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:32:58.552 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:32:58.552 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:32:58.554 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:58.552+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:58.555 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:32:58.655 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:58.653+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:58.677 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:32:58.677 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:32:58.677 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:32:58.677 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:32:58.686 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:32:58.692 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:32:58.693 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:32:58.711 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:32:58.711 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:32:58.728 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:32:58.728 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:32:58.744 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:32:58.745 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:32:58.756 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:58.754+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:58.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:32:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:32:58.779 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:32:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:32:58.795 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:32:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:32:58.802 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:58.803 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:58.812 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:32:58.812 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:32:58.813 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:32:58.866 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:58.864+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:58.920 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:58.920 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:58.928 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:32:58.970 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:58.968+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:59.059 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:32:59.057+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:32:59.077 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:59.077 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:59.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:32:59.160 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:59.158+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:59.233 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:59.233 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:59.242 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:32:59.252 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:32:59.287 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:59.284+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:59.350 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:59.351 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:59.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:32:59.361 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:32:59.359+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:32:59.396 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:32:59.462 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:59.460+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:59.473 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:59.473 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:59.483 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:32:59.567 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:32:59.565+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:32:59.632 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:59.632 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:59.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:32:59.664 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:32:59.662+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:32:59.766 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:32:59.764+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:32:59.788 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:59.789 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:59.798 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:32:59.799 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:32:59.812 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-09-15T21:32:59.812 INFO:tasks.mon_recovery:stopping all monitors 2024-09-15T21:32:59.812 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:32:59.812 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:32:59.825 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:32:59.825 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:32:59.825 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:32:59.866 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:32:59.867 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:32:59.867 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:32:59.867 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:32:59.864+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:32:59.910 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:32:59.910 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:32:59.919 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:32:59.919 INFO:tasks.mon_recovery:forming a minimal quorum for ['b', 'c', 'a'], then adding monitors 2024-09-15T21:32:59.919 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:32:59.920 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:32:59.921 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:32:59.923 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:32:59.923 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:32:59.923 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:32:59.963 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:32:59.963 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-09-15T21:32:59.964 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:00.005 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:00.003+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:00.068 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:00.066+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:00.070 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:00.070 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:00.080 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:00.169 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:00.167+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:00.228 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:00.228 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:00.237 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:00.243 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6] out_osds: [7, 5] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:33:00.243 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:33:00.243 INFO:tasks.thrashosds.thrasher:Adding osd 5 2024-09-15T21:33:00.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 5 2024-09-15T21:33:00.287 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:00.285+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:00.342 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:00.342 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:00.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:00.371 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:00.369+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:00.453 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:00.454 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:00.462 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:00.472 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:00.471+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:00.564 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:00.564 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:00.573 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:00.573 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:00.571+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:00.674 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:00.672+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:00.715 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:00.715 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:00.724 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:00.808 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:00.806+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:00.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:00.867 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:00.867 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:00.876 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:00.876 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:00.873+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:00.976 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:00.974+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:01.020 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:01.020 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:01.029 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:01.077 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:01.075+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:01.131 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:01.132 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:01.141 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:01.185 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:01.183+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:01.243 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:01.243 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:01.251 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:01.293 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:01.291+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:01.379 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:01.377+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:01.395 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:01.396 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:01.404 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:01.480 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:01.478+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:01.548 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:01.548 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:01.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:01.581 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:01.579+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:01.660 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:01.660 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:01.669 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:01.681 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:01.679+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:01.772 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:01.772 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:01.781 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:01.782 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:01.780+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:01.883 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:01.881+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:01.901 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:01.901 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:01.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:01.985 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:01.982+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:02.020 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:02.020 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:02.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:02.085 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:02.083+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:02.137 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:02.137 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:02.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:02.186 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:02.184+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:02.287 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:02.285+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:02.294 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:02.294 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:02.303 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:02.388 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:02.386+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:02.411 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:02.411 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:02.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:02.489 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:02.487+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:02.568 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:02.568 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:02.577 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:02.621 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:02.619+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:02.685 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:02.685 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:02.691 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:02.689+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:02.694 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:02.791 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:02.789+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:02.841 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:02.841 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:02.850 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:02.895 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:02.893+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:02.958 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:02.958 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:02.967 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:03.008 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:03.006+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:03.094 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:03.093+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:03.115 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:03.115 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:03.123 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:03.208 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:03.206+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:03.272 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:03.272 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:03.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:03.297 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:03.295+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:03.398 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:03.396+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:03.429 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:03.429 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:03.437 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:03.498 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:03.497+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:03.554 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:03.554 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:03.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:03.607 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:03.605+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:03.667 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:03.667 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:03.676 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:03.717 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:03.715+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:03.801 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:03.799+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:03.819 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:03.819 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:03.828 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:03.902 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:03.900+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:03.970 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:03.970 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:03.979 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:04.002 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:04.001+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:04.081 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:04.090 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:04.103 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:04.102+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:04.193 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:04.194 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:04.203 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:04.204 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:04.202+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:04.305 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:04.303+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:04.305 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:04.306 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:04.315 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:04.408 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:04.404+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:04.457 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:04.458 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:04.466 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:04.507 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:04.506+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:04.568 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:04.568 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:04.577 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:04.621 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:04.619+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:04.680 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:04.680 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:04.689 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:04.709 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:04.707+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:04.811 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:04.809+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:04.832 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:04.832 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:04.842 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:04.912 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:04.911+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:04.943 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:04.944 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:04.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:05.013 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:05.011+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:05.072 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:05.073 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:05.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:05.123 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:05.121+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:05.215 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:05.213+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:05.233 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:05.234 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:05.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:05.267 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:05.268 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:05.282 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:33:05.287 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:05.287 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:05.316 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:05.314+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:05.319 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:05.319 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:05.346 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:05.347 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:05.356 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.912952700999995, 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.357 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:05.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.912039668999995, 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:05.366 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.806171397, 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:05.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:05.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.805759127000002, 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:05.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:05.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.805310958000007, 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.376 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:05.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.804472610000005, 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:05.381 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:05.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.803556143999998, 2024-09-15T21:33:05.385 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:05.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:05.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.353128573999996, 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:05.394 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.352681814999997, 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:05.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.399 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.352254572999996, 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.400 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:05.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.351787623999996, 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:05.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:05.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.351419081000003, 2024-09-15T21:33:05.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:05.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.413 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.351002011000006, 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:05.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.348350995000004, 2024-09-15T21:33:05.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:05.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:05.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.347653665999999, 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:05.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:05.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:05.427 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.064328771999996, 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:05.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:05.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:05.435 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.060338841999993, 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:05.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.05968258, 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:05.440 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:05.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:05.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.055229022000006, 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:05.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.052496266999995, 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:05.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.454 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.454 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:05.455 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:05.455 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:05.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:05.457 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:05.457 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:05.457 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:05.457 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:05.458 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:05.458 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:05.459 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:33:05.433+0000 7fba73fff640 0 --2- 172.21.15.19:0/620521753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba5c004800 0x7fba741d9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:33:05.459 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:05.453+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:05.544 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":36,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:05.556 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:05.554+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:05.557 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-09-15T21:33:05.558 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:33:05.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:33:05.600 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:33:05.601 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-09-15T21:33:05.601 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:05.607 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:05.607 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:05.607 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:05.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:05.642 INFO:tasks.ceph.mon.b.smithi192.stderr:2024-09-15T21:33:05.640+0000 7f5585534640 -1 mon.b@1(leader).osd e39 definitely_dead 0 2024-09-15T21:33:05.659 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:05.656+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:05.704 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-09-15T21:33:05.702+0000 7f9553314640 -1 mon.a@0(leader).osd e39 definitely_dead 0 2024-09-15T21:33:05.757 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:05.756+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:05.763 INFO:teuthology.orchestra.run.smithi019.stderr:marked in osd.5. 2024-09-15T21:33:05.777 INFO:tasks.thrashosds.thrasher:Added osd 5 2024-09-15T21:33:05.802 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:05.802 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:05.803 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:05.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:05.859 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:05.857+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:05.959 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:05.957+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 93.501042049999995, 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:05.960 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:05.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:05.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:05.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 92.495160658000003, 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:05.965 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:05.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:05.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:05.970 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:05.970 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:05.971 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:06.017 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:06.032 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-09-15T21:33:06.032 INFO:tasks.mon_recovery:stopping all monitors 2024-09-15T21:33:06.033 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:33:06.033 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:33:06.045 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:33:06.046 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:33:06.046 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:33:06.074 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:06.072+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:06.087 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:33:06.087 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:33:06.087 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:33:06.125 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:06.126 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:06.126 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:06.135 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:06.136 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:33:06.136 INFO:tasks.mon_recovery:forming a minimal quorum for ['b', 'c', 'a'], then adding monitors 2024-09-15T21:33:06.136 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:33:06.136 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:33:06.139 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:33:06.139 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:33:06.139 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:33:06.160 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:06.158+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:06.178 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:33:06.178 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-09-15T21:33:06.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:06.262 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:06.260+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:06.285 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:06.285 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:06.286 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:06.286 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:06.294 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:06.379 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:06.377+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:06.442 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 94.985267750000006, 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:06.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:06.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:06.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:06.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:06.448 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 9.3228773529999991, 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:06.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:06.452 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:06.453 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:06.453 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:06.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:06.498 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:06.496+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:06.565 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:06.563+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:06.604 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:06.604 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:06.605 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:06.605 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:06.613 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:06.666 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:06.664+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:06.762 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:06.762 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:06.762 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:06.766 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:06.764+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:06.770 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:06.868 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:06.866+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:06.918 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:06.918 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 9.7993987570000005, 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.919 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:06.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:06.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.922 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:06.923 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:06.924 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:06.924 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:06.927 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:06.971 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:06.969+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:07.032 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:07.032 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:07.032 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:07.032 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:07.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:07.070 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:07.068+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:07.171 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:07.169+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:07.183 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:07.183 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:07.183 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:07.183 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:07.183 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:07.191 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:07.273 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:07.271+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:07.294 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:07.294 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:07.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:07.294 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:07.295 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:07.298 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:33:07.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:07.374 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:07.372+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:07.406 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:07.406 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:07.406 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:07.406 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:07.415 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:07.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:07.477 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:07.472+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:07.519 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:07.519 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:07.519 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:07.528 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:07.612 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:07.610+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 96.218181039000001, 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:07.673 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:07.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:07.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:07.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:07.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:07.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:07.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:07.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:07.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:07.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:07.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:07.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:07.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:07.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:07.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:07.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:07.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:07.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:07.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:07.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 96.212041408999994, 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:07.686 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:07.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:07.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:07.690 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 96.208873233000006, 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:07.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:07.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:07.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:07.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:07.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:07.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:07.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:07.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:07.696 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.696 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.696 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.697 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:07.697 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:07.697 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 96.208327479999994, 2024-09-15T21:33:07.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:07.697 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:07.697 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:07.697 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:07.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:07.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:07.698 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:07.698 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:07.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.698 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:07.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:07.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:07.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:07.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:07.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:07.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:07.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:07.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:07.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:07.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:07.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:07.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:07.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:07.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:07.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:07.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:07.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:07.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:07.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:07.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:07.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:07.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.705 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.705 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.705 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:07.705 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:07.706 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.215689171999998, 2024-09-15T21:33:07.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:07.706 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:07.706 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:07.706 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:07.706 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:07.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:07.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:07.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:07.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:07.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.714 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.212396343999998, 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:07.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:07.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:07.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 94.209845573999999, 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:07.719 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:07.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:07.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:07.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:07.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:07.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:07.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:07.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:07.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:07.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:07.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:07.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:07.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:07.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:07.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:07.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:07.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:07.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:07.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:07.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:07.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:07.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:07.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:07.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:07.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:07.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:07.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:07.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:07.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:07.730 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 94.206444884999996, 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:07.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:07.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:07.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:07.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.735 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.735 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.735 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:07.735 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:07.735 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:07.736 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:07.737 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:07.734+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:07.837 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:07.835+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:07.879 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:07.880 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:07.880 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:07.880 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:07.889 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:07.938 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:07.936+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:07.991 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:07.991 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:07.992 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:07.992 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:08.000 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:08.039 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:08.037+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:08.103 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:08.103 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 45.188973609999998, 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:08.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.105 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:08.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.162636190999997, 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.108 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:08.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:08.112 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.159202923000002, 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:08.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.117 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:08.118 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:08.140 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:08.139+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:08.220 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:08.221 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:08.221 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:08.221 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:08.229 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:08.273 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:08.271+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:08.333 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:08.333 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:08.334 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:08.334 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:08.334 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:08.341 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:08.339+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:08.342 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:08.442 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:08.440+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:08.487 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:08.487 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 103.06661402899999, 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:08.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:08.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.04326918300001, 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.495 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:08.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:08.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.9419022, 2024-09-15T21:33:08.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:08.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.511 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.941718382, 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.941679537, 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.518 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:08.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:08.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.941627563, 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:08.524 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:08.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:08.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.941611835, 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.530 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:08.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:08.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.941593187, 2024-09-15T21:33:08.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:08.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:08.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:08.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.94153523599999, 2024-09-15T21:33:08.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:08.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:08.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:08.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.941517405, 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:08.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:08.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:08.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.941436379, 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:08.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:08.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:08.626 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.94133512400001, 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.627 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:08.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:08.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:08.633 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.94128849499999, 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.634 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:08.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:08.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.483944642, 2024-09-15T21:33:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:08.641 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:08.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:08.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.48387545200001, 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.648 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:08.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.483610196, 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:08.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:08.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:08.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:08.661 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.483535926, 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:08.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:08.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.668 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.48350702499999, 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.669 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.675 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.48349169700001, 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:08.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.48340740899999, 2024-09-15T21:33:08.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:08.684 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:08.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:08.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:08.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:08.691 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:08.691 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:08.693 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:08.691+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:08.693 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:08.793 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:08.792+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:08.814 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:08.814 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:08.814 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:08.814 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:08.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:08.895 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:08.893+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:08.932 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:08.932 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:08.932 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:08.932 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:08.932 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:08.941 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:08.995 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:08.993+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:09.050 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:09.050 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:09.050 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:09.050 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:09.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.606406275, 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.051 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:09.052 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.053 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.054 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.055 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.605493243, 2024-09-15T21:33:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.056 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.057 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.058 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:09.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:09.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.058 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.058 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:09.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:09.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:09.061 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.499624971, 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.063 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:09.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.065 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.499212701, 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.066 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:09.067 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.068 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:09.069 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.070 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.498764532, 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:09.071 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.072 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.072 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.072 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.072 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.072 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.072 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:09.073 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.073 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.073 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.073 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:09.073 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.073 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.073 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:09.074 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.075 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.076 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.49792618399999, 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.077 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.078 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.078 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.078 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.078 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.078 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:09.078 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.079 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.079 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.079 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:09.079 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.079 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.079 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.080 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:09.081 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.497009718, 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.084 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.085 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.086 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.086 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:09.086 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:09.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.086 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.086 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.086 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:09.086 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:09.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.087 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.087 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.087 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:09.087 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:09.087 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.087 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.046582148, 2024-09-15T21:33:09.088 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:09.089 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.089 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.089 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.089 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.089 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.089 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.089 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.090 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.091 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:09.091 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:09.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.091 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.091 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.092 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:09.092 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:09.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.092 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.092 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.092 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:09.092 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:09.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.093 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.093 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.093 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:09.093 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:09.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.093 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.093 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.094 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:09.095 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.046135389, 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.096 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.097 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.098 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.099 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:09.099 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:09.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.099 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.099 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.099 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:09.100 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:09.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.100 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.100 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.100 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:09.100 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.101 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.045708147, 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.102 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:09.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.104 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:09.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:09.106 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.045241198, 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.107 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.108 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:09.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:09.110 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.04487265500001, 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.112 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:09.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:09.114 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.115 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.116 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.04445558499999, 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.117 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.118 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:09.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.120 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:09.121 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.04180456900001, 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:09.122 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.123 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.124 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:09.124 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:09.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.124 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.124 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:09.124 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:09.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.125 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.125 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:09.125 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:09.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.126 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.126 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:09.126 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:09.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.126 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:09.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:09.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.127 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:09.128 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:09.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.128 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.128 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:09.128 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:09.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.129 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.129 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.129 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:09.129 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:09.129 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.04110724, 2024-09-15T21:33:09.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:09.130 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.130 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.130 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.130 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:09.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:09.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 101.757782346, 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:09.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:09.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.139 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 101.753792416, 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:09.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:09.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 101.753136154, 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:09.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.148 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 101.74868259599999, 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:09.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.153 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 101.745949841, 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:09.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:09.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:09.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.158 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.158 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.158 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.158 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:09.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:09.160 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:09.157+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:09.260 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:09.258+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:09.308 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:09.308 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:09.308 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:09.308 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:09.317 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:09.360 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:09.359+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:09.461 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:09.459+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:09.465 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:09.465 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:09.465 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:09.465 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:09.465 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:09.474 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:09.561 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:09.560+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:09.583 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:09.583 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:09.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:09.583 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:09.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.584 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:09.584 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:09.584 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 97.123757346999994, 2024-09-15T21:33:09.584 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:09.584 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.584 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.585 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.585 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.585 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.585 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:09.586 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.586 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.586 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.586 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:09.586 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.587 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.587 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.587 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:09.587 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:09.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.587 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.587 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.588 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:09.588 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:09.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.588 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.588 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.589 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:09.589 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:09.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.589 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.589 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.589 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:09.590 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:09.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.590 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.590 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.590 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:09.590 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:09.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.591 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.591 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.591 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:09.591 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:09.591 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.592 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.592 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.592 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:09.592 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:09.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.593 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.593 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.593 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:09.593 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:09.593 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.593 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 96.117875955000002, 2024-09-15T21:33:09.594 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:09.595 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:09.595 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:09.595 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:09.595 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:09.595 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.595 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:09.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.597 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.597 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:09.597 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:09.597 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:09.597 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.597 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.597 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:09.598 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:09.598 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:09.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.598 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.598 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:09.598 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:09.598 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:09.599 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.599 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.599 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:09.599 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:09.599 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:09.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.600 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.600 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:09.600 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:09.600 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:09.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.601 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.601 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:09.601 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:09.601 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:09.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.601 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.602 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:09.602 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:09.602 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:09.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:09.602 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:09.603 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:09.604 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:09.605 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:09.689 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:09.687+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:09.753 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:09.753 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:09.753 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:09.753 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:09.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:09.803 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:09.801+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:09.865 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:09.863+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:09.909 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:09.909 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:09.910 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:09.910 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:09.910 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:09.918 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:09.965 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:09.963+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:10.025 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:10.025 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 98.568187409999993, 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:10.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:10.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:10.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.029 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.030 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 12.905797013000001, 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.031 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:10.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:10.034 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:10.035 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:10.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:10.066 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:10.064+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:10.143 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:10.143 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:10.143 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:10.143 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:10.151 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:10.167 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:10.166+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:10.258 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:10.258 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:10.258 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:10.258 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:10.258 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:10.267 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:10.269 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:10.267+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:10.370 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:10.368+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:10.375 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 13.256596383, 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:10.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:10.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:10.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:10.380 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:10.384 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:10.471 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:10.469+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:10.500 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:10.500 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:10.501 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:10.501 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:10.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:10.572 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:10.570+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:10.654 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:10.654 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:10.655 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:10.655 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:10.655 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:10.663 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:10.673 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:10.671+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:10.767 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:10.767 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:10.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:10.767 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:10.768 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:10.774 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:10.772+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:10.776 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:10.778 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:33:10.778 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:33:10.779 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-15T21:33:10.779 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T21:33:10.779 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T21:33:10.779 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-15T21:33:10.875 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:10.873+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:10.878 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:10.878 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:10.878 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:10.878 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:10.886 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:10.890 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:10.890 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T21:33:10.890 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:10.976 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:10.974+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:10.989 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:10.989 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:10.990 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:10.998 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:11.077 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:11.075+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:11.141 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:11.141 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:11.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 99.686867246000006, 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:11.142 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:11.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:11.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.148 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 99.680727615999999, 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:11.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:11.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:11.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 99.677559439999996, 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:11.153 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:11.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:11.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:11.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:11.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 99.677013686999999, 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:11.159 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:11.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:11.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.684375379000002, 2024-09-15T21:33:11.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:11.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:11.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:11.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.681082551000003, 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:11.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:11.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:11.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:11.174 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.678531781000004, 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:11.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:11.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 97.675131092000001, 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:11.181 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:11.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:11.185 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:11.187 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:11.187 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:11.185+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:11.288 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:11.286+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:11.330 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:11.331 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:11.331 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:11.331 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:11.339 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:11.423 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:11.421+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:11.459 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:11.459 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:11.477 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:11.477 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:11.485 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:11.485 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:11.485 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:11.485 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:11.485 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:11.489 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:11.487+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:11.496 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:11.496 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:11.497 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:11.513 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:11.513 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:11.531 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:11.531 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:11.547 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:11.548 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:11.565 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:11.566 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:11.582 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:11.582 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:11.590 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:11.588+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:11.640 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:11.640 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:11.649 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:11.692 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:11.690+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:11.792 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:11.790+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:11.792 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:11.793 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:11.801 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:11.893 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:11.891+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:11.942 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:11.943 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:11.951 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:11.994 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:11.992+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:12.053 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:12.054 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:12.062 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:12.106 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:12.104+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:12.170 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:12.170 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:12.181 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:12.222 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:12.220+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:12.296 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:12.294+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:12.329 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:12.329 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:12.341 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:12.397 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:12.395+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:12.493 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:12.494 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:12.498 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:12.496+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:12.500 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:33:12.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:12.599 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:12.597+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:12.599 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":42,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:12.618 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-09-15T21:33:12.618 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:33:12.619 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:33:12.621 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:33:12.621 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-09-15T21:33:12.621 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:12.671 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:12.671 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:12.682 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:12.700 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:12.698+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:12.794 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:12.801 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:12.799+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:12.803 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:12.902 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:12.900+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:12.911 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:12.911 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:12.921 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:13.001 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:13.002 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:13.000+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:13.014 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-09-15T21:33:13.014 INFO:tasks.mon_recovery:removing mon b 2024-09-15T21:33:13.015 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:33:13.015 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:33:13.027 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:33:13.027 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-09-15T21:33:13.028 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:13.031 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:13.032 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:13.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:13.103 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:13.100+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:13.196 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:13.196 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:13.203 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:13.201+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:13.205 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:13.304 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:13.302+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:13.313 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:13.313 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:13.322 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:13.405 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:13.403+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:13.410 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:13.430 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:13.431 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:13.440 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:13.524 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:13.522+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:13.587 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:13.587 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:13.594 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:13.596 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:13.609 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:13.607+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:13.710 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:13.709+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:13.746 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:13.746 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:13.755 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:13.761 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:13.762 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:13.778 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:13.779 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:13.795 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:13.795 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:13.812 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:13.812 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:13.813 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:13.809+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:13.828 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:13.829 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:13.845 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:13.845 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:13.862 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:13.871 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:13.879 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:13.879 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:13.881 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:13.925 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:13.923+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:13.984 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:13.984 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:13.984 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:13.993 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:14.034 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:14.032+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:14.113 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:14.111+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:14.136 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:14.136 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:14.137 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:14.137 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:14.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:14.214 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:14.212+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:14.289 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:14.289 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:14.289 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:14.289 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:14.298 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:14.315 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:14.313+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:14.401 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:14.401 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:14.402 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:14.402 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:14.402 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:14.410 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:14.454 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:14.452+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 103.058190361, 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.513 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:14.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:14.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:14.519 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 103.05205073099999, 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:14.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 103.04888255500001, 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:14.524 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:14.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:14.528 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 103.04833680199999, 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:14.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.055698494, 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:14.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:14.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:14.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.539 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.052405666, 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:14.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.543 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.049854896, 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:14.544 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:14.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:14.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 101.046454207, 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:14.551 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:14.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:14.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:14.555 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.0222190690000001, 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:14.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:14.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.0192129040000002, 2024-09-15T21:33:14.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:14.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:14.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:14.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:14.565 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:14.565 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:33:14.566 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:14.567 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:14.564+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:14.667 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:14.665+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:14.711 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:14.712 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:14.712 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:14.712 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:14.720 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:14.804 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:14.802+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:14.862 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:14.862 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:14.863 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:14.863 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:14.863 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:14.868 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:14.866+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:14.870 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:14.969 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:14.967+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:15.012 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:15.013 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:15.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:15.013 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:15.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:15.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:15.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 52.098134936999998, 2024-09-15T21:33:15.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:15.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:15.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.017 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.071797517999997, 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:15.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.022 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 44.068364250000002, 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:15.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 3.5177626699999998, 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.027 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:15.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.031 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:15.032 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:15.079 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:15.077+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:15.138 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:15.138 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:15.138 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:15.138 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:15.146 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:15.187 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:15.185+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:15.272 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:15.270+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:15.288 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:15.288 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:15.288 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:15.288 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:15.288 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:15.297 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:15.373 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:15.371+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:15.402 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:15.402 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:15.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:15.402 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:15.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.402 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:15.402 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.98136726600001, 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:15.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:15.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:15.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:15.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.95802242000001, 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.410 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.413 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:15.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.416 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.856655437, 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.417 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:15.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:15.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.422 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.856471619, 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:15.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:15.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.856432774, 2024-09-15T21:33:15.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:15.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:15.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.8563808, 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:15.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:15.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.856365072, 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.442 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:15.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:15.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:15.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.85634642399999, 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:15.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:15.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.85628847300001, 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.458 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:15.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:15.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.856270642, 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.464 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:15.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:15.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.85618961599999, 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:15.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:15.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.856088361, 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:15.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:15.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.85604173199999, 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:15.483 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:15.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:15.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:15.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.489 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.398697879, 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:15.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.39862868900001, 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.496 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:15.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:15.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.398363433, 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:15.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:15.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.398289163, 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:15.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.39826026199999, 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.514 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:15.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:15.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.398244934, 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.520 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:15.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:15.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:15.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:15.526 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.39816064599999, 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:15.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:15.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:15.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:15.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:15.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:15.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.533 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:15.533 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:15.533 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:15.534 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:15.535 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:15.533+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:15.635 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:15.633+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:15.652 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:15.652 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:15.652 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:15.661 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:15.736 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:15.734+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:15.769 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:15.769 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:15.770 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:15.778 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:15.837 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:15.835+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:15.890 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:15.899 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5] out_osds: [7] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:33:15.899 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:33:15.899 INFO:tasks.thrashosds.thrasher:Adding osd 7 2024-09-15T21:33:15.899 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 7 2024-09-15T21:33:15.928 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:15.928 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 109.484623696, 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.929 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.931 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 109.483710664, 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:15.934 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.935 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.936 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.938 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 109.37784239200001, 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.939 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:15.940 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.941 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:15.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:15.943 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 109.37743012200001, 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.944 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:15.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.946 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.947 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 109.376981953, 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:15.948 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.950 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:15.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 109.376143605, 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.953 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:15.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 109.375227139, 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:15.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.962 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.924799569, 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:15.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:15.967 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.92435281, 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:15.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:15.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.923925568, 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:15.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:15.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.923458619, 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:15.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:15.981 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.92309007599999, 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:15.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.922673006, 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.986 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:15.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:15.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.990 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.92002199, 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:15.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:15.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:15.995 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.919324661, 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:15.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:15.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.635999767, 2024-09-15T21:33:16.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.632009837, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:16.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:16.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.009 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.63135357500001, 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:16.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.014 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.626900017, 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:16.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.624167262, 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:16.019 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:16.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.024 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.024 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:16.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:16.027 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:16.024+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:16.126 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:16.124+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:16.176 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:16.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:16.194 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:33:16.192+0000 7fb5caffd640 0 --2- 172.21.15.19:0/3197387475 >> v2:172.21.15.19:3300/0 conn(0x7fb5cc0aa6e0 0x7fb5cc146b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:33:16.227 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:16.226+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:16.302 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:16.302 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:16.303 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:16.303 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:16.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:16.328 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:16.326+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:16.419 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.419 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 103.96033576000001, 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:16.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.954454368, 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:16.424 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:16.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.428 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.429 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.429 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:16.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:16.431 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:16.431 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:16.428+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:16.532 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:16.529+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:16.581 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:16.581 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:16.581 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:16.581 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:16.590 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:16.632 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:16.630+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:16.698 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:16.698 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:16.698 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:16.698 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:16.698 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:16.707 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:16.751 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:16.749+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 105.356456464, 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:16.814 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:16.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.694066067000001, 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:16.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:16.822 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 5.3240321579999996, 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:16.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:16.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.826 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.827 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.827 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:16.827 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:16.827 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:16.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:16.843 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:16.844 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:16.841+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:16.935 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:16.933+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:16.950 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:16.951 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:16.951 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:16.951 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:16.959 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:17.036 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:17.034+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:17.106 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:17.106 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:17.106 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:17.107 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:17.107 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:17.115 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:17.136 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:17.135+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:17.222 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:17.222 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:17.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:17.222 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:17.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 20.102797673000001, 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:17.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:17.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:17.225 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:17.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:17.227 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:17.230 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:17.237 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:17.235+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:17.338 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:17.336+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:17.347 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:17.347 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:17.347 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:17.347 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:17.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:17.438 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:17.436+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:17.501 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:17.501 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:17.501 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:17.501 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:17.501 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:17.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:17.539 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:17.537+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:17.612 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:17.612 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:17.613 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:17.613 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:17.622 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:17.665 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:17.663+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:17.724 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:17.724 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:17.724 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:17.724 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:17.732 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:17.740 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:17.739+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:17.842 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:17.840+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:17.875 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:17.876 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:17.876 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:17.885 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:17.943 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:17.941+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:18.027 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.57307179199999, 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.028 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:18.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:18.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:18.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.566932162, 2024-09-15T21:33:18.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:18.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:18.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:18.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.563763986, 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.040 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:18.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:18.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 106.563218233, 2024-09-15T21:33:18.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:18.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:18.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.048 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.570579925, 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:18.049 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:18.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:18.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.567287097, 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:18.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:18.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:18.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.56473632700001, 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.060 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:18.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:18.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 104.561335638, 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.067 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:18.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5371005000000002, 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:18.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:18.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.5340943349999998, 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:18.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.080 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:18.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:18.082 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:18.080+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:18.183 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:18.181+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:18.225 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:18.226 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:18.226 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:18.226 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:18.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:18.284 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:18.282+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:18.336 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:18.336 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:18.336 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:18.337 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:18.337 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:18.345 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:18.386 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:18.384+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:18.486 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:18.484+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:18.487 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:18.487 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:18.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 55.57331113, 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:18.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:18.492 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.546973711, 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:18.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:18.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.543540442999998, 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:18.497 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:18.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 6.992938863, 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:18.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.505 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:18.506 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:18.591 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:18.589+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:18.651 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:18.651 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:18.651 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:18.660 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:18.701 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:18.699+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:18.789 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:18.787+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:18.803 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:18.803 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:18.804 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:18.804 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:18.812 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:18.889 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:18.887+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:18.916 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:18.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 113.49561415399999, 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:18.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:18.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:18.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:18.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:18.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.47226930799999, 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.927 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:18.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:18.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.370902325, 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:18.933 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:18.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:18.939 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.37071850700001, 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:18.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.945 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.370679662, 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:18.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.951 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.370627688, 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:18.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:18.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:18.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.37061196000001, 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.958 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:18.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.963 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.370593312, 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.964 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:18.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:18.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.37053536099999, 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:18.970 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:18.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:18.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:18.976 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.37051753, 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:18.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:18.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.982 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.370436504, 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:18.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:18.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.988 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.37033524899999, 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:18.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:18.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:18.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.37028862, 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.995 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:18.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:18.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:18.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.912944767, 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:19.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:19.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:19.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.91287557699999, 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:19.007 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:19.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:19.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:19.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.912610321, 2024-09-15T21:33:19.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:19.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:19.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:19.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:19.019 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.912536051, 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:19.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:19.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:19.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:19.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.91250715, 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:19.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:19.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:19.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.91249182200001, 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:19.032 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:19.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:19.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.912407534, 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:19.038 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:19.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:19.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:19.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:19.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:19.046 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:19.043+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:19.147 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:19.145+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:19.166 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:19.166 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:19.166 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:19.166 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:19.176 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:19.247 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:19.245+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:19.284 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:19.285 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:19.285 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:19.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:19.348 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:19.346+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:19.441 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:19.441 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.997796846, 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:19.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:19.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.996883814, 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:19.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:19.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:19.451 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.89101554200001, 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:19.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.89060327200001, 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:19.456 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:19.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.460 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.890155103, 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.461 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:19.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.465 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.889316755, 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.468 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:19.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.888400289, 2024-09-15T21:33:19.471 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:19.472 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:19.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.475 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.437972719, 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.476 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:19.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:19.480 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.43752596, 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:19.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.483 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:19.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.437098718, 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:19.486 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.487 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:19.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:19.490 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.436631769, 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:19.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.493 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:19.495 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.43626322599999, 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.496 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:19.497 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.498 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:19.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.435846156, 2024-09-15T21:33:19.500 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:19.501 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.502 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:19.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.504 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.43319514, 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.505 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.506 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.507 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:19.508 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.509 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.432497811, 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.510 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.511 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:19.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:19.514 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.149172917, 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:19.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.517 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:19.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:19.519 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.145182987, 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:19.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:19.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.14452672500001, 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.524 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:19.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:19.527 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.528 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.140073167, 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:19.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.534 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:19.534 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:19.536 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.137340412, 2024-09-15T21:33:19.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:19.536 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:19.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.540 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.541 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.541 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:19.542 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:19.543 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:19.541+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:19.643 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:19.641+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:19.691 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:19.692 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:19.692 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:19.692 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:19.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:19.744 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:19.742+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:19.807 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:19.808 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:19.808 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:19.808 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:19.808 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:19.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:19.845 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:19.843+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:19.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:19.946 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:19.944+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:19.966 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 107.50792630399999, 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:19.971 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 106.502044912, 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:19.972 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:19.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:19.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.977 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.977 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.977 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:19.977 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:19.977 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:19.978 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:20.062 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:20.060+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:20.126 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:20.126 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:20.126 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:20.126 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:20.135 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:20.149 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:20.147+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:20.229 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":46,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:20.249 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:20.248+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:20.282 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:20.283 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:20.283 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:20.283 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:20.283 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:20.292 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:20.376 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:20.374+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:20.439 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.982279487, 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:20.440 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:20.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:20.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 23.31988909, 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:20.445 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:20.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 8.9498551810000002, 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:20.449 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:20.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:20.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:20.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.453 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:20.453 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 3.221367689, 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:20.454 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:20.456 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:20.458 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:20.460 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:20.460 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:20.458+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:20.560 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:20.558+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:20.607 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:20.608 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:20.608 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:20.608 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:20.616 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:20.661 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:20.659+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:20.723 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:20.723 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:20.724 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:20.724 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:20.724 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:20.732 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:20.762 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:20.760+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:20.863 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:20.861+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:20.881 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:20.881 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:20.881 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:20.881 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 23.761674202000002, 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:20.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:20.884 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:20.885 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:20.886 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:20.886 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:20.886 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.886 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:20.886 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.886 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:20.886 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:20.886 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:20.889 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:20.965 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:20.963+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:21.004 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:21.004 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:21.004 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:21.005 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:21.013 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:21.066 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:21.064+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:21.117 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:21.118 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:21.118 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:21.118 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:21.118 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:21.126 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:21.167 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:21.165+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:21.268 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:21.266+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:21.270 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:21.270 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:21.270 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:21.279 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:21.373 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:21.367+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:21.422 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:21.422 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:21.423 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:21.423 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:21.431 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:21.472 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:21.470+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:21.571 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:21.569+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:21.573 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:21.576 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:21.576 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:21.576 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:21.576 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:21.582 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:21.672 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:21.670+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.232212558, 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:21.687 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:21.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:21.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:21.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.22607292799999, 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:21.694 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.698 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.22290475200001, 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:21.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:21.702 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.22235899899999, 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:21.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:21.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:21.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.229720691, 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:21.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:21.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:21.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.226427863, 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:21.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.223877093, 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.719 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:21.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:21.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:21.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.220476404, 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:21.726 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:21.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:21.730 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.196241265999999, 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.731 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:21.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:21.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.734 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.193235101000001, 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:21.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:21.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:21.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:21.739 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:21.739 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:21.740 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:21.773 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:21.771+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:21.842 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:21.842 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:21.843 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:21.843 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:21.851 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:21.895 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:21.893+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:21.953 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:21.953 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:21.953 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:21.962 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:22.003 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:22.001+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:22.075 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:22.072+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:22.106 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:22.106 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:22.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:22.106 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 59.192129164999997, 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:22.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.165791745999996, 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:22.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.115 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 51.162358478000002, 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.117 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:22.118 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.611756897999999, 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:22.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:22.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.124 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.124 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.124 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.124 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:22.125 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:22.209 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:22.207+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:22.267 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:22.267 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:22.268 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:22.276 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:22.276 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:22.274+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:22.377 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:22.374+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:22.419 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:22.420 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:22.420 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:22.420 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:22.420 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:22.428 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:22.513 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:22.511+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 117.151927333, 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.573 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:22.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:22.579 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.128582487, 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.580 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:22.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:22.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:22.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.027215504, 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.587 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:22.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.593 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.027031686, 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:22.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:22.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:22.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:22.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:22.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.02699284099999, 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:22.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:22.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.611 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.611 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:22.611 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.02694086699999, 2024-09-15T21:33:22.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:22.611 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:22.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:22.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:22.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:22.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.026925139, 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:22.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:22.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.624 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.02690649100001, 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.625 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:22.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:22.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.02684854, 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.631 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:22.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:22.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.026830709, 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.637 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:22.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.02674968300001, 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:22.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.026648428, 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:22.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:22.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.02660179900001, 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.655 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:22.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:22.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.56925794599999, 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:22.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:22.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:22.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.569188756, 2024-09-15T21:33:22.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:22.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:22.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:22.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.5689235, 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.674 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:22.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:22.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:22.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.56884923, 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.680 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:22.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:22.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.685 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.568820329, 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.686 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:22.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.568805001, 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.692 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:22.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:22.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 115.568720713, 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:22.698 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:22.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:22.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:22.704 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:22.705 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:22.706 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:22.704+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:22.706 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:22.807 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:22.805+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:22.827 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:22.827 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:22.827 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:22.837 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:22.907 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:22.905+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:22.945 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:22.945 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:22.945 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:22.953 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:23.038 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:23.036+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:23.103 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:23.103 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.65935895200001, 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.105 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:23.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.65844592000001, 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:23.108 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.110 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.111 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.112 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.552577648, 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.113 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:23.114 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.115 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:23.116 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.552165378, 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:23.118 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.119 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.120 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.121 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:23.122 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.551717209, 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.123 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:23.125 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.126 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:23.127 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.550878861, 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:23.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:23.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.54996239499999, 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.133 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:23.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.137 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.09953482500001, 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:23.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:23.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.142 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.09908806599999, 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:23.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.09866082400001, 2024-09-15T21:33:23.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:23.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:23.152 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.09819387500001, 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:23.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.097825332, 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.157 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:23.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.160 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.161 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.097408262, 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:23.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:23.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:23.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:23.166 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.094757246, 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:23.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.170 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.094059917, 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:23.171 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:23.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:23.175 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 115.81073502300001, 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.176 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:23.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.180 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 115.806745093, 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:23.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 115.806088831, 2024-09-15T21:33:23.185 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.189 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 115.801635273, 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:23.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.194 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 115.79890251800001, 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.195 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.196 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:23.198 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.199 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.200 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.200 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:23.201 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:23.202 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:23.199+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:23.243 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:23.302 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:23.300+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:23.351 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:23.351 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:23.352 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:23.352 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:23.360 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:23.402 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:23.401+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:23.503 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:23.501+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:23.508 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:23.509 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:23.509 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:23.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:23.603 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:23.601+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:23.665 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:23.665 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 111.206654043, 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.666 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:23.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.668 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.669 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.670 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 110.20077265099999, 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.671 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:23.672 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.673 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:23.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:23.675 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:23.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:23.704 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:23.702+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:23.805 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:23.803+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:23.825 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:23.825 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:23.826 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:23.834 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:23.919 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:23.917+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:23.981 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:23.981 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:23.981 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:23.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:24.007 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:24.006+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:24.108 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:24.107+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 112.679539992, 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 27.017149594999999, 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 12.647115685999999, 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 6.9186281940000001, 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:33:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:33:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:33:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:33:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:24.154 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:24.155 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:24.210 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:24.208+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:24.262 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:24.263 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:24.263 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:24.271 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:24.315 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:24.313+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:24.378 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:24.378 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:24.378 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:24.386 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:24.411 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:24.409+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:24.512 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:24.511+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:24.535 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:24.535 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:24.535 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:24.535 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:24.535 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.535 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:24.535 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:24.535 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 27.415876096000002, 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:24.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:24.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:24.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:24.537 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:24.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:24.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:24.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:24.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:24.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:24.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:24.540 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:24.541 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:24.541 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:24.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:24.614 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:24.612+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:24.660 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:24.661 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:24.661 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:24.661 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:24.670 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:24.715 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:24.713+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:24.773 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:24.773 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:24.773 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:24.773 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:24.773 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:24.782 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:24.823 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:24.821+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:24.917 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:24.915+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:24.924 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:24.924 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:24.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:24.924 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:24.924 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:24.933 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:25.018 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:25.016+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:25.037 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:25.037 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:25.037 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:25.037 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:25.049 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:25.119 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:25.117+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:25.153 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:25.153 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:25.154 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:25.154 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:25.154 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:25.162 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:25.220 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:25.218+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 113.84998850300001, 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:25.305 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:25.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:25.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:25.312 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 113.843848873, 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:25.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:25.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 113.840680697, 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:25.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:25.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 113.840134944, 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.321 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:25.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.325 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.847496636, 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:25.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:25.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:25.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.332 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.844203808, 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:25.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:25.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.841653038, 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.337 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:25.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:25.343 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 111.838252349, 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:25.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.814017210999999, 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:25.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.352 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.811011046000001, 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:25.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.357 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:25.358 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:25.359 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:25.356+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:25.461 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:25.459+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:25.501 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:25.501 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:25.501 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:25.501 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:25.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:25.562 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:25.560+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:25.612 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:25.612 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:25.612 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:25.612 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:25.612 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:25.621 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:25.664 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:25.662+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:25.766 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:25.764+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:25.767 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 62.852710602000002, 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:25.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.826373183000001, 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.772 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:25.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.776 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.822939914999999, 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:25.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:25.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 14.272338335000001, 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:25.781 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:25.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:25.784 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 2.8527557109999999, 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:25.785 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:25.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:33:25.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:25.789 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:25.790 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:25.867 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:25.865+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:25.893 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:25.893 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:25.893 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:25.893 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:25.902 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:25.968 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:25.966+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:26.045 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:26.046 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:26.046 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:26.046 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:26.046 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:26.054 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:26.069 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:26.068+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:26.159 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:26.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.73838731399999, 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.160 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:26.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:26.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:26.167 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.71504246800001, 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:26.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:26.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.173 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.613675485, 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.174 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:26.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.177 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:26.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.61349166700001, 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.180 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.181 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:26.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:26.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.613452822, 2024-09-15T21:33:26.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:26.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:26.192 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.613400848, 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.193 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:26.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:26.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:26.199 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.61338512, 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:26.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:26.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:26.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.613366472, 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.206 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:26.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:26.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.61330852099999, 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.213 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:26.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.61329069, 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.219 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:26.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.613209664, 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.225 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:26.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:26.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:26.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.61310840900001, 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:26.231 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.232 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:26.233 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.234 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.236 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.237 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.61306178, 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.238 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:26.239 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.240 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:26.241 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.242 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:26.243 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:26.244 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.155717927, 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.245 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.246 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.15564873700001, 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.251 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:26.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:26.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.257 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.155383481, 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:26.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:26.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.263 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.155309211, 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.264 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:26.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:26.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.15528030999999, 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:26.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:26.276 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.15526498200001, 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.277 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:26.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:26.283 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.15518069399999, 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:26.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:26.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:26.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:26.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:26.290 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:26.290 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:26.291 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:26.290+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:26.291 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:26.392 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:26.390+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:26.414 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:26.414 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:26.414 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:26.414 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:26.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:26.493 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:26.491+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:26.530 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:26.530 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:26.530 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:26.530 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:26.530 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:26.538 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:26.623 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:26.621+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:26.688 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:26.688 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:26.688 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:26.688 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:26.688 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.688 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:26.688 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:26.688 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 120.24421438500001, 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.689 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:26.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.691 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:26.692 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 120.24330135300001, 2024-09-15T21:33:26.693 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.694 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:26.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.696 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 120.137433081, 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.698 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.699 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:26.700 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.701 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.702 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:26.702 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:26.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.702 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.702 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:26.702 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:26.703 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 120.137020811, 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.704 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:26.706 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.707 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 120.136572642, 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.709 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:26.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:26.711 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:26.712 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 120.135734294, 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.713 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.715 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:26.716 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.717 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 120.134817828, 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.718 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.719 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.720 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.721 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.722 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.68439025799999, 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.723 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:26.724 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:26.725 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.726 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.68394349899999, 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.727 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.728 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:26.729 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:26.730 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.731 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.68351625699999, 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.732 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.736 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.68304930799999, 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:26.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.682680765, 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:26.741 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:26.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:26.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:26.746 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.682263695, 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:26.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.749 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.679612679, 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:26.751 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:26.752 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.753 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:26.755 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.67891535, 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.756 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.757 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:26.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.760 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.39559045599999, 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:26.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.391600526, 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:26.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.390944264, 2024-09-15T21:33:26.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:26.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:26.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.386490706, 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:26.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:26.779 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.38375795100001, 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:26.780 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:26.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.784 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:26.785 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:26.785 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:26.786 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:26.787 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:26.784+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:26.886 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:26.885+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:26.935 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:26.935 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:26.935 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:26.935 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:26.944 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:26.987 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:26.985+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:27.088 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:27.086+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:27.091 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:27.092 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:27.092 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:27.092 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:27.092 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:27.100 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:27.189 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:27.187+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 114.790008216, 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:27.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:27.251 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.252 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:27.253 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 113.784126824, 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:27.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:27.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:27.256 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.257 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:27.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:27.290 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:27.288+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:27.366 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:27.366 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:27.366 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:27.366 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:27.375 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:27.391 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:27.390+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:27.483 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:27.483 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:27.492 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:27.492 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:27.490+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:27.593 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:27.591+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:27.640 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:27.640 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 116.18299270599999, 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:27.641 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:27.642 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.643 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:27.644 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.645 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 30.520602309000001, 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.646 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:27.647 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:27.648 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:27.649 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:27.650 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 16.150568400000001, 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:27.651 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:27.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:27.653 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:27.654 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 10.422080908, 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.655 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:33:27.656 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.657 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:33:27.658 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.659 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:27.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:27.659 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:27.659 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:27.660 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:27.694 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:27.692+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:27.768 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:27.768 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:27.768 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:27.768 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:27.776 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:27.793 INFO:tasks.ceph.mon.a.smithi019.stderr:2024-09-15T21:33:27.791+0000 7fc37e3c3640 -1 mon.a@0(leader).osd e42 definitely_dead 0 2024-09-15T21:33:27.794 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:27.793+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:27.816 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:27.816 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:27.835 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:27.835 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:27.852 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:27.852 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:27.869 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:27.869 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:27.886 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:27.887 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:27.891 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:27.891 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:27.895 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:27.893+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:27.901 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:27.905 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:27.906 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:27.923 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:27.923 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:27.940 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:27.940 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:27.995 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:27.993+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:28.014 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:28.014 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:28.023 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:28.072 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":50,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:28.085 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-09-15T21:33:28.085 INFO:tasks.mon_recovery:causing some monitor log activity 2024-09-15T21:33:28.086 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '1 of 30' 2024-09-15T21:33:28.096 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:28.094+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:28.139 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:28.140 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:28.148 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:28.233 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:28.231+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:28.293 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:28.293 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:28.298 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:28.296+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:28.302 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:28.399 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:28.397+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:28.446 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:28.447 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:28.455 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:28.500 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:28.498+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:28.597 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:28.598 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:28.601 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:28.600+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:28.606 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:28.702 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:28.700+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:28.708 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:28.708 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:28.717 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:28.800 INFO:teuthology.orchestra.run.smithi019.stderr:marked in osd.7. 2024-09-15T21:33:28.801 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:33:28.803 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:28.801+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:28.810 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '2 of 30' 2024-09-15T21:33:28.818 INFO:tasks.thrashosds.thrasher:Added osd 7 2024-09-15T21:33:28.821 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:28.821 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:28.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:28.904 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:28.902+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:28.972 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:28.972 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:28.980 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:29.024 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:29.022+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:29.083 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:29.083 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:29.092 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:29.133 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:29.131+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:29.206 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:29.204+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:29.235 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:29.235 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:29.244 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:29.307 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:29.305+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:29.388 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:29.388 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:29.397 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:29.441 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:29.439+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:29.499 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:29.499 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:29.508 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:29.509 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:29.507+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:29.609 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:29.607+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:29.651 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:29.652 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:29.660 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:29.710 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:29.708+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:29.773 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:29.773 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:29.782 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:29.808 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '3 of 30' 2024-09-15T21:33:29.810 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:29.809+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:29.911 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:29.910+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:29.953 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:29.957 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:29.957 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:29.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:30.052 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:30.050+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:30.083 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:30.083 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:30.100 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:30.100 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:30.114 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:30.112+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:30.116 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:30.117 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:30.122 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:30.122 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:30.122 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.678799008, 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.123 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.125 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.126 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:30.127 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.677885976, 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:30.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:30.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.572017704, 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:30.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.57160543400001, 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:30.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:30.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.571157265, 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.143 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:30.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.147 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.57031891699999, 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:30.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.151 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:30.152 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.569402451, 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:30.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:30.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.118974881, 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:30.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.160 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:30.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:30.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:30.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:30.162 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.162 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.162 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.118528122, 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.165 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:30.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:30.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:30.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.11810088, 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.170 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:30.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:30.174 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.117633931, 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:30.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:30.179 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.11726538800001, 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.180 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:30.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.184 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.116848318, 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:30.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.11419730199999, 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.189 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.191 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:30.193 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 123.113499973, 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:30.195 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:30.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:30.197 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 122.830175079, 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.199 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.200 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:30.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.202 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 122.826185149, 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.204 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.205 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.206 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 122.825528887, 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:30.208 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.209 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.210 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.211 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 122.821075329, 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.212 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.213 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.215 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:30.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 122.818342574, 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:30.217 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:30.218 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:30.219 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.220 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.222 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.222 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:30.223 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:30.223 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:30.224 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:30.224 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:30.224 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:30.225 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:30.225 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:30.226 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:30.226 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:30.227 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:30.221+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:30.323 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:30.322+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:30.371 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:30.371 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:30.371 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:30.371 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:30.380 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:30.424 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:30.423+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:30.488 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:30.488 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:30.488 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:30.488 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:30.488 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:30.497 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:30.538 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:30.536+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:30.627 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:30.625+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:30.644 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:30.644 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:30.644 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:30.644 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:30.644 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.644 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 118.185534164, 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.645 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:30.646 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.647 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:30.648 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 117.179652772, 2024-09-15T21:33:30.649 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:30.650 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.651 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.652 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:30.653 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.654 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:30.654 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:30.654 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:30.655 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:30.739 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:30.737+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:30.804 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:30.804 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:30.805 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:30.805 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:30.811 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '4 of 30' 2024-09-15T21:33:30.814 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:30.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:33:30.829 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:30.827+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:30.929 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:30.928+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:30.967 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:30.967 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:30.967 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:30.967 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:30.967 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:30.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:31.030 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:31.029+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:31.127 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:31.127 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:31.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:31.127 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:31.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.669437195, 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:31.128 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:31.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 34.007046797999998, 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:31.132 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:31.133 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:31.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:31.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 19.637012889000001, 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:31.136 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 13.908525397, 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:31.140 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:33:31.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:31.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:31.144 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:31.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:31.146 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:31.144+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:31.247 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:31.245+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:31.253 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:31.253 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:31.254 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:31.254 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:31.262 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:31.348 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:31.346+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:31.410 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:31.411 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:31.411 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:31.411 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:31.411 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:31.419 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:31.449 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:31.447+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:31.549 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:31.547+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:31.567 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:31.567 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:31.567 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:31.567 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:31.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 34.448432887999999, 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:31.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:31.571 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:31.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.572 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:31.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.572 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:31.572 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:31.572 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:31.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:31.650 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:31.648+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:31.693 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:31.693 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:31.693 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:31.693 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:31.702 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:31.751 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:31.749+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:31.804 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:33:31.813 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '5 of 30' 2024-09-15T21:33:31.846 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:31.846 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:31.846 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:31.846 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:31.846 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:31.855 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:31.862 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:31.860+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:31.952 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:31.950+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:31.958 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:31.958 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:31.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:31.958 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:31.958 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:31.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:32.053 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:32.052+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:32.069 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:32.069 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:32.070 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:32.070 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:32.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:32.155 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:32.153+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:32.180 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:32.181 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:32.181 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:32.190 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:32.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:32.274 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:32.272+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:32.334 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:32.334 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:32.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:32.334 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:32.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.334 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.87951148800001, 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:32.335 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:32.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:32.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:32.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.873371858, 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:32.342 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:32.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:32.346 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.870203682, 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:32.347 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:32.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.351 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:32.352 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:32.352 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.869657929, 2024-09-15T21:33:32.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:32.352 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.352 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.352 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:32.352 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.877019621, 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:32.357 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:32.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:32.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.363 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 119.873726793, 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:32.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:32.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:32.368 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 118.871176023, 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:32.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:32.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:32.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.375 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 118.867775334, 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:32.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.843540195999999, 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:32.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 20.840534031000001, 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:32.385 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:32.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:32.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:32.389 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:32.390 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:32.391 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:32.391 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:32.392 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:32.392 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:32.393 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:32.393 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:32.393 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:32.389+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:32.402 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:32.402 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:32.419 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:32.419 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:32.436 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:32.437 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:32.453 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:32.453 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:32.471 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:32.471 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:32.491 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:32.490+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:32.537 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:32.546 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:32.630 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:32.629+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:32.690 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:32.690 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:32.693 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:32.692+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:32.698 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:32.794 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:32.792+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:32.814 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '6 of 30' 2024-09-15T21:33:32.842 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:32.842 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:32.851 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:32.898 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:32.896+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:32.954 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:32.963 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:32.995 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:32.993+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:33.065 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:33.065 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:33.074 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:33.096 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:33.094+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:33.177 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:33.177 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:33.185 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:33.197 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:33.196+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:33.294 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:33.294 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:33.298 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:33.296+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:33.302 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:33.399 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:33.398+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:33.450 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:33.451 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:33.459 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:33.503 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:33.501+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:33.567 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:33.567 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:33.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:33.601 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:33.600+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:33.702 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:33.701+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:33.724 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:33.724 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:33.733 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:33.802 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:33.800+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:33.810 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '7 of 30' 2024-09-15T21:33:33.819 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:33:33.819 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:33:33.819 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-15T21:33:33.819 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T21:33:33.819 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T21:33:33.820 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-15T21:33:33.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:33:33.842 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:33.842 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:33.851 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:33.935 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:33.933+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:33.935 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:33.935 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T21:33:33.935 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:34.003 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:34.001+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:34.004 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:34.004 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:34.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:34.105 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:34.104+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:34.162 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:34.163 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:34.172 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:34.206 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:34.204+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:34.280 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:34.289 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:34.307 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:34.305+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:34.397 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:34.397 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:34.406 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:34.407 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:34.405+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:34.482 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:34.508 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:34.506+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:34.514 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:34.514 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:34.523 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:34.609 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:34.609 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:34.610 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:34.607+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:34.626 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:34.626 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:34.643 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:34.643 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:34.660 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:34.673 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:34.673 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:34.673 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:34.674 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:34.674 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:34.676 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:34.677 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:34.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:34.694 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:34.694 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:34.710 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:34.708+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:34.711 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:34.712 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:34.729 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:34.729 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 37.671689266999998, 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:34.791 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:34.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:34.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.794 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:34.795 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:34.799 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:34.805 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:33:34.812 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:34.810+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:34.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '8 of 30' 2024-09-15T21:33:34.908 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:34.909 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:34.909 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:34.909 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:34.913 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:34.911+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:34.917 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:35.014 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:35.012+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:35.020 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:35.020 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:35.020 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:35.020 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:35.020 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:35.028 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:35.115 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:35.113+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:35.173 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:35.173 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:35.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:35.173 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:35.173 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:35.182 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:35.226 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:35.224+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:35.285 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:35.286 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:35.286 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:35.294 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:35.317 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:35.316+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:35.418 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:35.416+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:35.436 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:35.436 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:35.436 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:35.437 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:35.437 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:35.445 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:35.519 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:35.517+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:35.587 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:35.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.132750511, 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:35.588 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:35.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:35.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:35.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:35.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.126610881, 2024-09-15T21:33:35.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:35.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.123442705, 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:35.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:35.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 124.122896952, 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.608 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.13025864399999, 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.609 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:35.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:35.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:35.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:35.615 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 123.12696581599999, 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:35.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.124415046, 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:35.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:35.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:35.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.12101435700001, 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:35.627 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:35.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.096779218999998, 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:35.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:35.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.093773054, 2024-09-15T21:33:35.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:35.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:35.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:35.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.640 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:35.641 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:35.641 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:35.642 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:35.643 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:35.640+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:35.742 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:35.741+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:35.785 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:35.785 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:35.785 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:35.793 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:35.825 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '9 of 30' 2024-09-15T21:33:35.866 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:35.864+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:35.896 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:35.897 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:35.897 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:35.905 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:35.949 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:35.947+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:36.009 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:36.009 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.094683674999999, 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:36.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:36.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:36.014 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.068346255999998, 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:36.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.064912988000003, 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:36.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.514311408000001, 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:36.023 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.094728784000001, 2024-09-15T21:33:36.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:33:36.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:33:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:33:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.032 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.032 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:36.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:36.045 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:36.043+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:36.145 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:36.143+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:36.177 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:36.177 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:36.177 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:36.178 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:36.186 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:36.270 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:36.268+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:36.329 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:36.329 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:36.330 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:36.330 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:36.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:36.379 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:36.377+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:36.448 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:36.447+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:36.482 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 131.061775219, 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 130.03843037300001, 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.93706338999999, 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.93687957200001, 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.936840727, 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.509 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:36.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.936788753, 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:36.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.521 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.93677302500001, 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:36.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:36.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.527 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.936754377, 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:36.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:36.533 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.936696426, 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.534 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:36.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:36.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:36.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:36.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.93667859499999, 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:36.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:36.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.93659756900001, 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.547 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:36.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:36.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.93649631400001, 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:36.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:36.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.93644968500001, 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:36.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:36.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:36.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.565 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.47910583199999, 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:36.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.567 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:36.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.571 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.47903664200001, 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.572 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:36.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:36.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.47877138600001, 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.578 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:36.580 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:36.583 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.47869711600001, 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:36.584 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:36.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:36.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.478668215, 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:36.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.592 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:36.593 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:36.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.47865288700001, 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.597 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:36.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:36.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:36.603 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 129.478568599, 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:36.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:36.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:36.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:36.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.609 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:36.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:36.610 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:36.611 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:36.612 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:36.609+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:36.712 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:36.710+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:36.726 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:36.727 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:36.727 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:36.736 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:36.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:36.822 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:33:36.823 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:36.821+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:36.826 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '10 of 30' 2024-09-15T21:33:36.847 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:36.847 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:36.848 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:36.857 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:36.915 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:36.913+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:36.915 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:36.915 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:36.935 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:36.936 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:36.954 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:36.954 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:36.975 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:36.976 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:36.993 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:36.993 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:37.010 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:37.010 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:37.016 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:37.014+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:37.016 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:37.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:37.036 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:37.036 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:37.054 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:37.054 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:37.117 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:37.115+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:37.146 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:37.146 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:37.155 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:37.217 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:37.215+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:37.302 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:37.302 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:37.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:37.319 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:37.317+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:37.418 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:37.418 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:37.419 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:37.417+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:37.426 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:37.520 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:37.518+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:37.575 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:37.575 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:37.584 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:37.622 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:37.620+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:37.691 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:37.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:37.743 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:37.741+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:37.808 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:37.809 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:37.817 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:33:37.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:37.827 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '11 of 30' 2024-09-15T21:33:37.827 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:37.825+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:37.925 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:37.923+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:37.938 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:37.938 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:37.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:38.025 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:38.023+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:38.066 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:38.066 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:38.075 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:38.126 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:38.124+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:38.222 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:38.227 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:38.226+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:38.231 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:38.328 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:38.327+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:38.347 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:38.347 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:38.356 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:38.430 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:38.428+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:38.459 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:38.460 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:38.468 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:38.530 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:38.529+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:38.572 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:38.572 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:38.580 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:38.664 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:38.662+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:38.723 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:38.723 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:38.733 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:38.733 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:38.731+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:38.832 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '12 of 30' 2024-09-15T21:33:38.833 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:38.831+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:38.876 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:38.876 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:38.885 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:38.935 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:38.933+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:38.945 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:33:38.945 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:33:38.945 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:39.028 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:39.028 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:39.036 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:39.034+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:39.037 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:39.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:39.137 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:39.135+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:39.141 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:39.141 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:39.149 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:39.197 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:39.198 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:39.216 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:39.216 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:39.232 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:39.233 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:39.238 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:39.236+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:39.248 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:39.248 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:39.252 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:39.252 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:39.258 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:33:39.259 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:33:39.261 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:39.265 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:39.266 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:39.271 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-15T21:33:38.630791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220260,"kb_used_data":4028,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754460,"statfs":{"total":773094113280,"available":772868567040,"internally_reserved":0,"allocated":4124672,"data_stored":2687540,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.826061"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:33:35.175792+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:33:35.175792+0000","last_peered":"2024-09-15T21:33:35.175792+0000","last_clean":"2024-09-15T21:33:35.175792+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:33:35.175792+0000","last_undegraded":"2024-09-15T21:33:35.175792+0000","last_fullsized":"2024-09-15T21:33:35.175792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":65,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:33:32.917766+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:33:32.917766+0000","last_peered":"2024-09-15T21:33:32.917766+0000","last_clean":"2024-09-15T21:33:32.917766+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:33:32.917766+0000","last_undegraded":"2024-09-15T21:33:32.917766+0000","last_fullsized":"2024-09-15T21:33:32.917766+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:33:34.808735+0000","last_change":"2024-09-15T21:33:29.803626+0000","last_active":"2024-09-15T21:33:34.808735+0000","last_peered":"2024-09-15T21:33:34.808735+0000","last_clean":"2024-09-15T21:33:34.808735+0000","last_became_active":"2024-09-15T21:33:29.803416+0000","last_became_peered":"2024-09-15T21:33:29.803416+0000","last_unstale":"2024-09-15T21:33:34.808735+0000","last_undegraded":"2024-09-15T21:33:34.808735+0000","last_fullsized":"2024-09-15T21:33:34.808735+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:13.277699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":66,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:33:32.917786+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:33:32.917786+0000","last_peered":"2024-09-15T21:33:32.917786+0000","last_clean":"2024-09-15T21:33:32.917786+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:33:32.917786+0000","last_undegraded":"2024-09-15T21:33:32.917786+0000","last_fullsized":"2024-09-15T21:33:32.917786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":56,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:33:35.175792+0000","last_change":"2024-09-15T21:33:11.494030+0000","last_active":"2024-09-15T21:33:35.175792+0000","last_peered":"2024-09-15T21:33:35.175792+0000","last_clean":"2024-09-15T21:33:35.175792+0000","last_became_active":"2024-09-15T21:33:11.490555+0000","last_became_peered":"2024-09-15T21:33:11.490555+0000","last_unstale":"2024-09-15T21:33:35.175792+0000","last_undegraded":"2024-09-15T21:33:35.175792+0000","last_fullsized":"2024-09-15T21:33:35.175792+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:41:16.816826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":68,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:33:37.463930+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:33:37.463930+0000","last_peered":"2024-09-15T21:33:37.463930+0000","last_clean":"2024-09-15T21:33:37.463930+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:33:37.463930+0000","last_undegraded":"2024-09-15T21:33:37.463930+0000","last_fullsized":"2024-09-15T21:33:37.463930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:33:34.808695+0000","last_change":"2024-09-15T21:33:30.398919+0000","last_active":"2024-09-15T21:33:34.808695+0000","last_peered":"2024-09-15T21:33:34.808695+0000","last_clean":"2024-09-15T21:33:34.808695+0000","last_became_active":"2024-09-15T21:33:30.398643+0000","last_became_peered":"2024-09-15T21:33:30.398643+0000","last_unstale":"2024-09-15T21:33:34.808695+0000","last_undegraded":"2024-09-15T21:33:34.808695+0000","last_fullsized":"2024-09-15T21:33:34.808695+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:19.700973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":23,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:33:31.805711+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:33:31.805711+0000","last_peered":"2024-09-15T21:33:31.805711+0000","last_clean":"2024-09-15T21:33:31.805711+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:33:31.805711+0000","last_undegraded":"2024-09-15T21:33:31.805711+0000","last_fullsized":"2024-09-15T21:33:31.805711+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":21,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:33:34.808726+0000","last_change":"2024-09-15T21:33:29.811291+0000","last_active":"2024-09-15T21:33:34.808726+0000","last_peered":"2024-09-15T21:33:34.808726+0000","last_clean":"2024-09-15T21:33:34.808726+0000","last_became_active":"2024-09-15T21:33:29.803834+0000","last_became_peered":"2024-09-15T21:33:29.803834+0000","last_unstale":"2024-09-15T21:33:34.808726+0000","last_undegraded":"2024-09-15T21:33:34.808726+0000","last_fullsized":"2024-09-15T21:33:34.808726+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:54.379060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476766,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":552960,"data_stored":736737,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476767,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":202338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476767,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":202338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476767,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":552960,"data_stored":731266,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542174,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":492,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":503808,"data_stored":202357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":516096,"data_stored":207828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":202338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27516,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344324,"statfs":{"total":96636764160,"available":96608587776,"internally_reserved":0,"allocated":499712,"data_stored":202338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:39.283 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:39.283 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:39.300 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:39.300 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:39.316 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:39.317 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:39.365 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:39.425 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:39.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 76.450583491000003, 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:39.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.424246072000003, 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:39.430 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.420812803999993, 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:39.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.870211223999998, 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.440 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:39.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:39.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:39.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:39.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:39.444 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 16.450628600000002, 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.445 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:33:39.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:33:39.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:33:39.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.449 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.449 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.449 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:39.450 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:39.368+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:39.451 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:39.449+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:39.451 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:39.551 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:39.550+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:39.554 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:39.554 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:39.555 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:39.555 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:39.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:39.651 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:39.649+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:39.666 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:39.666 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:39.666 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:39.666 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:39.666 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:39.675 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:39.753 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:39.751+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:39.819 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.39888540800001, 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.820 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:39.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:39.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:39.827 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.375540562, 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:39.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.27417357900001, 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:39.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:39.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:39.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.273989761, 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:39.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.27395091599999, 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.846 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:39.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:39.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.27389894199999, 2024-09-15T21:33:39.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:39.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:39.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.858 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.27388321399999, 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:39.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.864 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.27386456599999, 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.865 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:39.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:39.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.27380661500001, 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:39.871 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:39.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.273788784, 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:39.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:39.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:39.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.273707758, 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.884 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:39.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.273606503, 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.890 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:39.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:39.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:39.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.273559874, 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:39.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:39.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.816216021, 2024-09-15T21:33:39.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:39.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:39.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.816146831, 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.910 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:39.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:39.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:39.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.81588157499999, 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.916 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:39.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:39.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.922 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.81580730499999, 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:39.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:39.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.928 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.81577840400001, 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.929 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:39.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.815763076, 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:39.935 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:39.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.941 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.81567878800001, 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:39.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:39.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:39.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:39.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:39.947 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.948 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.948 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.948 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:39.948 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:39.948 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:39.948 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:33:39.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '13 of 30' 2024-09-15T21:33:39.951 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:39.947+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:39.951 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:40.050 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:40.048+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:40.110 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:40.110 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:40.111 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:40.121 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:40.151 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:40.149+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:40.229 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:40.230 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:40.230 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:40.238 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:40.252 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:40.250+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.90516030699999, 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:40.349 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:40.353 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.90424727499999, 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:40.354 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.356 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:40.358 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.79837900300001, 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:40.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.79796673300001, 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:40.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.366 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:40.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.797518564, 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:40.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:40.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:40.373 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.796680216, 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:40.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.79576374999999, 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:40.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:40.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.34533618, 2024-09-15T21:33:40.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:40.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:40.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.344889421, 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:40.388 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:40.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.392 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.344462179, 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:40.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.395 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:40.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.34399522999999, 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:40.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.400 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:40.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:40.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.34362668700001, 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.405 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:40.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:40.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.34320961700001, 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:40.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:40.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.340558601, 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:40.414 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:40.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.33986127200001, 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:40.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.056536378, 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.05254644799999, 2024-09-15T21:33:40.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:40.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.05189018600001, 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:40.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:40.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.04743662800001, 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:40.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:40.442 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.044703873, 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:40.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:40.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.447 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.447 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:40.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:40.449 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:40.446+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:40.550 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:40.547+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:40.556 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:40.556 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:40.556 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:40.565 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:40.651 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:40.649+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:40.672 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:40.673 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:40.673 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:40.681 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:40.765 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:40.763+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:40.819 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:33:40.831 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:40.831 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 128.37224470199999, 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.832 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.833 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:40.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:40.835 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 127.36636331, 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:40.836 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:40.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:40.838 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:40.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.840 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.841 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.841 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:40.841 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:40.841 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:40.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:40.843 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '14 of 30' 2024-09-15T21:33:40.883 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:40.881+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:40.953 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:40.952+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:40.993 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:40.994 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:41.005 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:41.089 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:41.087+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:41.154 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:41.154 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:41.154 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:41.154 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:41.154 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:41.155 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:41.153+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:41.163 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:41.256 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:41.254+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:41.312 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:41.312 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:41.313 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:41.313 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:41.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.313 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:41.313 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:41.314 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 129.85478286599999, 2024-09-15T21:33:41.314 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:41.314 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:41.314 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:41.314 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:41.314 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:41.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.315 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:41.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:41.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:41.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:41.316 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:41.316 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:41.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:41.317 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:41.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:41.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.317 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:41.317 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:41.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:41.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:41.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:41.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:41.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:41.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:41.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:41.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:41.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:41.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:41.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:41.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:41.322 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 44.192392468999998, 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:41.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:41.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.327 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:41.327 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:41.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:41.327 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.327 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:41.327 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.328 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:41.328 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:41.328 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 29.822358560000001, 2024-09-15T21:33:41.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:41.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:41.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:41.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:41.332 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 24.093871067999999, 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:41.333 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:41.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:33:41.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:41.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:41.337 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:41.338 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:41.340 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:41.357 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:41.355+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:41.458 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:41.456+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:41.491 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:41.492 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:41.504 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:41.512 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:41.512 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:41.530 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:41.530 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:41.549 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:41.549 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:41.559 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:41.557+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:41.566 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:41.566 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:41.583 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:41.584 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:41.600 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:41.601 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:41.613 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:41.613 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:41.618 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:41.618 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:41.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:41.635 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:41.636 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:41.660 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:41.658+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:41.742 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:41.743 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:41.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:41.760 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:41.758+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:41.839 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '15 of 30' 2024-09-15T21:33:41.867 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:41.867 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:41.876 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:41.880 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:41.878+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:41.962 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:41.960+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:41.981 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:41.981 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:41.990 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:42.074 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:42.072+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:42.132 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:42.132 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:42.141 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:42.164 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:42.162+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:42.265 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:42.263+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:42.284 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:42.284 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:42.293 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:42.366 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:42.365+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:42.396 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:42.396 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:42.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:42.467 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:42.465+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:42.507 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:42.507 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:42.516 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:42.568 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:42.567+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:42.619 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:42.619 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:42.628 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:42.669 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:42.668+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:42.730 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:42.730 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:42.739 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:42.783 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:42.781+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:42.842 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:33:42.843 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:42.843 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:42.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '16 of 30' 2024-09-15T21:33:42.851 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:42.893 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:42.889+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:42.972 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:42.970+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:42.996 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:42.996 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:43.004 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:43.089 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:43.087+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:43.149 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:43.149 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:43.157 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:43.174 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:43.171+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:43.275 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:43.272+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:43.302 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:43.302 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:43.311 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:43.376 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:43.374+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:43.419 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:43.419 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:43.427 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:43.477 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:43.475+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:43.535 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:43.535 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:43.544 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:43.578 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:43.576+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:43.648 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:43.651 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:43.651 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:43.660 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:43.701 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:43.698+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:43.775 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:43.775 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:43.780 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:43.777+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:43.793 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:43.794 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:43.810 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:43.810 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:43.811 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:43.811 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:43.811 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:43.811 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:43.820 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:43.826 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:43.827 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:43.840 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:33:43.847 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:43.848 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:43.850 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '17 of 30' 2024-09-15T21:33:43.865 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:43.865 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:43.885 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:43.885 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:43.886 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:43.882+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:43.902 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:43.903 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:43.941 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:43.941 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:43.941 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:43.941 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:43.941 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:43.949 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:43.991 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:43.988+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:44.084 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:44.082+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:44.098 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 131.63998410799999, 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:44.099 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.101 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.102 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:44.103 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 130.634102716, 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:44.104 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:44.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:44.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.107 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.107 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:44.107 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:44.107 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:44.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.107 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.108 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:44.108 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:44.108 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:44.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.108 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:44.108 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:44.109 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.110 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.110 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.110 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.110 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.110 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:44.111 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:44.185 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:44.182+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:44.220 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:44.220 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:44.220 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:44.221 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:44.229 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:44.273 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:33:44.273 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:33:44.313 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:44.310+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:44.376 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:44.376 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:44.377 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:44.377 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:44.377 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:44.385 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:44.426 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:44.423+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:44.489 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:44.487+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:44.533 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:44.533 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:44.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:44.533 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 133.07596409000001, 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:44.534 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.535 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:44.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:44.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:44.538 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 47.413573693000004, 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:44.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:44.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 33.043539783999996, 2024-09-15T21:33:44.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:44.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 27.315052292000001, 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:33:44.546 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:44.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:33:44.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:33:44.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:33:44.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.551 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:44.552 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:44.590 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:44.588+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:44.660 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:44.660 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:44.660 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:44.660 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:44.668 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:44.713 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:44.710+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:44.776 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:44.776 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:44.776 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:44.776 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:44.785 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:44.826 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:44.823+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:44.846 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '18 of 30' 2024-09-15T21:33:44.930 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:44.927+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:44.935 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:44.935 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 47.816546952000003, 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:44.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:44.937 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.938 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:44.939 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:44.940 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:44.945 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:44.992 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:44.989+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:45.060 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:45.060 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:45.060 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:45.060 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:45.069 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:45.093 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:45.090+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:45.173 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:45.173 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:45.173 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:45.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:45.193 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:45.191+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:45.284 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:45.284 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:45.284 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:45.292 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:45.293 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:45.292+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:45.395 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:45.393+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:45.395 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:45.395 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:45.396 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:45.404 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:45.496 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:45.494+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:45.547 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:45.547 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:45.548 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:45.548 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:45.556 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:45.597 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:45.595+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:45.698 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:45.696+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:45.700 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:45.700 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:45.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:45.700 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:45.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.24577467500001, 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.701 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:45.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.703 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:45.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:45.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:45.707 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.239635045, 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.709 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:45.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.236466869, 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.712 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:45.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 134.23592111599999, 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:45.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:45.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.243282808, 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:45.720 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:45.721 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:45.722 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.723 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:45.724 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:45.725 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.726 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.23998997999999, 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:45.728 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.729 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:45.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.23743920999999, 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:45.731 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:45.733 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.734 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.735 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:45.736 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.737 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 132.234038521, 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.738 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:45.739 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.740 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.741 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.209803383000001, 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.742 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:45.743 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:45.744 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.745 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.206797217999998, 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:45.746 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:45.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:45.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:45.750 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:45.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:45.799 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:45.797+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:45.848 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '19 of 30' 2024-09-15T21:33:45.858 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:33:45.894 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:45.894 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:45.894 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:45.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:45.919 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:45.933 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:45.930+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:46.000 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:45.997+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:46.006 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:46.006 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:46.006 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:46.015 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:46.068 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:46.068 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:46.085 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:46.085 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:46.101 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:46.102 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:46.099+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:46.102 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:46.118 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:46.118 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:46.118 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:46.118 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:46.118 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 83.203846217000006, 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.119 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:46.120 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.121 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.122 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 75.177508798000005, 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.123 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:46.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:46.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 75.174075529999996, 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:46.127 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:46.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:46.131 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 34.623473949999998, 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:46.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 23.203891326000001, 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:46.137 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:33:46.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:46.141 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:46.142 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:46.142 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:46.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:46.160 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:46.160 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:46.176 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:46.176 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:46.193 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:46.193 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:46.202 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:46.200+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:46.209 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:46.209 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:46.248 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:46.249 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:46.257 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:46.303 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:46.301+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:46.399 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:46.399 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:46.403 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:46.402+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:46.408 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:46.505 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:46.503+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:46.551 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:46.551 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:46.560 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:46.606 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:46.604+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:46.671 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:46.671 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:46.680 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:46.724 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:46.721+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:46.787 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:46.788 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:46.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:46.837 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:46.834+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:46.839 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:33:46.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '20 of 30' 2024-09-15T21:33:46.909 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:46.907+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:46.947 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:46.947 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:46.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:47.010 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:47.008+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:47.065 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:47.074 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:47.111 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:47.109+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:47.181 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:47.181 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:47.191 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:47.211 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:47.209+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:47.299 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:47.308 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:47.312 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:47.310+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:47.413 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:47.411+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:47.415 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:47.415 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:47.424 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:47.513 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:47.512+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:47.531 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:47.532 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:47.540 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:47.614 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:47.611+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:47.648 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:47.648 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:47.657 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:47.741 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:47.738+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:47.805 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:47.805 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:47.814 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:47.816 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:47.814+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:47.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '21 of 30' 2024-09-15T21:33:47.916 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:47.914+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:48.003 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:48.003 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:48.012 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:48.017 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:48.015+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:48.117 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:48.115+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:48.121 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:48.121 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:48.130 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:48.219 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:48.216+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:48.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:48.245 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:48.245 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:48.254 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:48.320 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:48.317+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:48.345 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:48.345 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:48.359 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:48.359 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:48.362 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:48.362 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:48.367 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:48.378 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:48.378 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:48.394 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:48.395 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:48.412 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:48.413 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:48.421 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:48.418+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:48.429 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:48.429 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:48.447 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:48.447 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:48.463 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:48.464 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:48.474 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:48.475 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:48.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:48.475 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:48.475 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:48.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:48.527 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:48.525+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:48.586 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:48.587 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:48.587 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:48.587 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:48.596 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:48.622 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:48.620+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:48.723 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:48.720+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:48.739 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:48.739 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:48.739 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:48.739 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:48.740 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:48.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:48.823 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:48.821+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:48.851 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:48.851 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:48.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:48.851 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:48.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.851 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:48.851 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.39651028899999, 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:48.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:48.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:48.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.858 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.39037065900001, 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:48.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:48.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:48.863 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.38720248300001, 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:48.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.867 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 137.38665673, 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:48.868 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:48.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:48.872 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 136.39401842199999, 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:48.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 136.390725594, 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.880 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:48.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:48.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 135.388174824, 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:48.885 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:48.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:48.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:48.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:48.891 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 135.38477413499999, 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:48.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.360538996999999, 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:48.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.357532831999997, 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:48.901 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:48.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:48.905 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:48.906 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:33:48.907 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '22 of 30' 2024-09-15T21:33:48.908 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:48.953 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:48.950+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:49.011 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:49.012 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:49.012 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:49.012 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:49.020 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:49.062 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:49.060+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:49.125 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:49.123+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:49.163 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:49.164 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:49.164 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:49.164 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:49.164 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:49.172 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:49.227 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:49.225+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:49.274 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:33:49.274 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:33:49.274 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T21:33:49.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T21:33:49.275 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:49.275 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 86.360500740000006, 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.334163321000005, 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:49.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:49.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 78.330730052999996, 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.285 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:49.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.780128472999998, 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:49.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:49.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 26.360545849000001, 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:49.294 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:33:49.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:33:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:33:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.298 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.298 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:49.299 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:49.342 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:49.339+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:49.429 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:49.427+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:49.443 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:49.444 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:49.444 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:49.444 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:49.453 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:49.530 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:49.527+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:49.596 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:49.596 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:49.597 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:49.597 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:49.597 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:49.602 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:33:49.602 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:49.602 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 50, 2024-09-15T21:33:49.602 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:33:49.602 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2024-09-15T21:31:12.329884+0000", 2024-09-15T21:33:49.602 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2024-09-15T21:33:46.829934+0000", 2024-09-15T21:33:49.602 INFO:teuthology.orchestra.run.smithi019.stdout: "last_up_change": "2024-09-15T21:31:25.414443+0000", 2024-09-15T21:33:49.602 INFO:teuthology.orchestra.run.smithi019.stdout: "last_in_change": "2024-09-15T21:33:28.789828+0000", 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_num": 5801984, 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_set": [ 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "noscrub", 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "pglog_hardlimit", 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "purged_snapdirs", 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_deletes", 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "sortbitwise" 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_version": 5, 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:33:49.603 INFO:teuthology.orchestra.run.smithi019.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_snapshot": "", 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_max": 2, 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd": 8, 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "require_min_compat_client": "luminous", 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "min_compat_client": "luminous", 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "require_osd_release": "squid", 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_crimson": false, 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "pools": [ 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 1, 2024-09-15T21:33:49.604 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": ".mgr", 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:22.622713+0000", 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool", 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:33:49.605 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 1, 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 1, 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 1, 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 1, 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 1, 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:33:49.606 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "18", 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 0, 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 0, 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:33:49.607 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:33:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:33:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:33:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:33:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:33:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:33:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:33:49.608 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:33:49.609 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:33:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:33:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:33:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:33:49.612 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:33:49.613 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "options": { 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_max": 32, 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_min": 1 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": {} 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.614 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 8, 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 8, 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 0.25, 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 2, 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 2, 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 0.98379135131835938, 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-15T21:33:49.615 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 2, 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": "rbd", 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:30.313202+0000", 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 8193, 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:33:49.616 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 8, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 8, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 8, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 8, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 8, 2024-09-15T21:33:49.617 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "22", 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:33:49.618 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:33:49.619 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:33:49.619 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:33:49.619 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 2, 2024-09-15T21:33:49.619 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 22, 2024-09-15T21:33:49.619 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:33:49.619 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:33:49.619 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:33:49.620 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:33:49.621 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "options": {}, 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd": {} 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 1.9900000095367432, 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 1.9900000095367432, 2024-09-15T21:33:49.622 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 0.87999999523162842, 2024-09-15T21:33:49.623 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 1.75, 2024-09-15T21:33:49.623 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 1.75, 2024-09-15T21:33:49.625 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 0.98379135131835938, 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osds": [ 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "f7eb9035-e8c2-4d90-a198-2bed78d5a3b5", 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:49.626 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:49.627 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:49.627 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:49.627 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:49.627 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:33:49.627 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 27, 2024-09-15T21:33:49.627 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:49.627 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:49.628 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:49.628 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.629 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6824", 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6825", 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.630 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6826", 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6827", 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.631 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6830", 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.632 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6831", 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.633 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6828", 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6829", 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6825/368705522", 2024-09-15T21:33:49.634 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6827/368705522", 2024-09-15T21:33:49.635 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6831/368705522", 2024-09-15T21:33:49.635 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6829/368705522", 2024-09-15T21:33:49.635 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:49.635 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:49.635 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:49.635 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.635 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a", 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:49.636 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 36, 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6816", 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.637 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6817", 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6818", 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.638 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6819", 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6822", 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:49.639 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6823", 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.640 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6820", 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6821", 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6817/3888640163", 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6819/3888640163", 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6823/3888640163", 2024-09-15T21:33:49.641 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6821/3888640163", 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "180bae67-0f64-45a0-9a58-f4b8ff2b90a5", 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:49.642 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 36, 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.643 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6808", 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6809", 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:49.644 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6810", 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6811", 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.645 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6814", 2024-09-15T21:33:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.646 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6815", 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6812", 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.647 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.648 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.648 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6813", 2024-09-15T21:33:49.648 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:49.648 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.648 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.648 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6809/1831747042", 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6811/1831747042", 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6815/1831747042", 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6813/1831747042", 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d05cec44-e228-4297-98c8-53e2ad91b37c", 2024-09-15T21:33:49.649 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:49.650 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:49.650 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:49.650 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:49.650 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:49.651 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6802", 2024-09-15T21:33:49.652 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6803", 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.653 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6806", 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6807", 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.654 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6804", 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6805", 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.655 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6801/3658047863", 2024-09-15T21:33:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6803/3658047863", 2024-09-15T21:33:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6807/3658047863", 2024-09-15T21:33:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6805/3658047863", 2024-09-15T21:33:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:49.656 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "5d7097a2-8165-4885-a356-f8e67851be5d", 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 0.870330810546875, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:33:49.657 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6816", 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6817", 2024-09-15T21:33:49.658 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6818", 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.659 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6819", 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.660 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6822", 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6823", 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6820", 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6821", 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6817/1988554297", 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6819/1988554297", 2024-09-15T21:33:49.662 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1988554297", 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1988554297", 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f", 2024-09-15T21:33:49.663 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 14, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 40, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:49.664 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6824", 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6825", 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.665 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6826", 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6827", 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:49.666 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6830", 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.667 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6831", 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.668 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6828", 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6829", 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6825/1071217311", 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6827/1071217311", 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6831/1071217311", 2024-09-15T21:33:49.669 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6829/1071217311", 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612", 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:49.670 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 43, 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6800", 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.671 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6801", 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6802", 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:49.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6803", 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6806", 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6807", 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.674 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6804", 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6805", 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6801/1069316293", 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6803/1069316293", 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6807/1069316293", 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6805/1069316293", 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.676 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:33:49.677 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "fb77f072-c3ae-4914-81bd-473d30016320", 2024-09-15T21:33:49.677 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:49.677 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:49.677 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:49.677 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 43, 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6808", 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:49.678 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6809", 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:49.679 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6810", 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6811", 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.680 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6814", 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6815", 2024-09-15T21:33:49.681 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6812", 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:49.682 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6813", 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6809/3119270077", 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6811/3119270077", 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6815/3119270077", 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6813/3119270077", 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.683 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:49.684 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_xinfo": [ 2024-09-15T21:33:49.684 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.684 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:33:49.684 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:49.684 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.795411+0000", 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:49.685 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.139237+0000", 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.562583+0000", 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:49.686 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.687 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.687 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:33:49.687 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:49.687 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:49.687 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:49.687 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:49.687 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:49.687 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.363254+0000", 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.344635+0000", 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:49.688 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.323050+0000", 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:33:49.689 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.381780+0000", 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:49.690 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.049941+0000", 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap": [ 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:49.691 INFO:teuthology.orchestra.run.smithi019.stdout: "pgid": "2.2", 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: "osds": [ 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: 6, 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: 5 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_items": [], 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_primaries": [], 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_temp": [], 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_temp": [], 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklist": {}, 2024-09-15T21:33:49.692 INFO:teuthology.orchestra.run.smithi019.stdout: "range_blocklist": {}, 2024-09-15T21:33:49.693 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profiles": { 2024-09-15T21:33:49.693 INFO:teuthology.orchestra.run.smithi019.stdout: "default": { 2024-09-15T21:33:49.693 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-failure-domain": "osd", 2024-09-15T21:33:49.693 INFO:teuthology.orchestra.run.smithi019.stdout: "k": "2", 2024-09-15T21:33:49.693 INFO:teuthology.orchestra.run.smithi019.stdout: "m": "1", 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: "plugin": "jerasure", 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: "technique": "reed_sol_van" 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps_queue": [], 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: "new_removed_snaps": [], 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: "new_purged_snaps": [], 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_node_flags": {}, 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: "device_class_flags": {}, 2024-09-15T21:33:49.694 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": { 2024-09-15T21:33:49.695 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_enabled": false, 2024-09-15T21:33:49.695 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_bucket_count": 0, 2024-09-15T21:33:49.695 INFO:teuthology.orchestra.run.smithi019.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:33:49.695 INFO:teuthology.orchestra.run.smithi019.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:33:49.695 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_bucket": 0 2024-09-15T21:33:49.695 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:49.695 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:49.697 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:49.699 INFO:tasks.thrashosds.thrasher:j is {'epoch': 50, 'fsid': '13019d66-d4b6-4e2a-8322-38bcb4210bcf', 'created': '2024-09-15T21:31:12.329884+0000', 'modified': '2024-09-15T21:33:46.829934+0000', 'last_up_change': '2024-09-15T21:31:25.414443+0000', 'last_in_change': '2024-09-15T21:33:28.789828+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T21:31:22.622713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9837913513183594, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:31:30.313202+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9837913513183594, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': 'f7eb9035-e8c2-4d90-a198-2bed78d5a3b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6824', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6825', 'nonce': 368705522}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6826', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6827', 'nonce': 368705522}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6830', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6831', 'nonce': 368705522}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6828', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6829', 'nonce': 368705522}]}, 'public_addr': '172.21.15.19:6825/368705522', 'cluster_addr': '172.21.15.19:6827/368705522', 'heartbeat_back_addr': '172.21.15.19:6831/368705522', 'heartbeat_front_addr': '172.21.15.19:6829/368705522', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd2bcecd4-0bb0-4b38-b395-8b6dbb48f53a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6816', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6817', 'nonce': 3888640163}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6818', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6819', 'nonce': 3888640163}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6822', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6823', 'nonce': 3888640163}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6820', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6821', 'nonce': 3888640163}]}, 'public_addr': '172.21.15.19:6817/3888640163', 'cluster_addr': '172.21.15.19:6819/3888640163', 'heartbeat_back_addr': '172.21.15.19:6823/3888640163', 'heartbeat_front_addr': '172.21.15.19:6821/3888640163', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '180bae67-0f64-45a0-9a58-f4b8ff2b90a5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6808', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6809', 'nonce': 1831747042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6810', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6811', 'nonce': 1831747042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6814', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6815', 'nonce': 1831747042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6812', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6813', 'nonce': 1831747042}]}, 'public_addr': '172.21.15.19:6809/1831747042', 'cluster_addr': '172.21.15.19:6811/1831747042', 'heartbeat_back_addr': '172.21.15.19:6815/1831747042', 'heartbeat_front_addr': '172.21.15.19:6813/1831747042', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd05cec44-e228-4297-98c8-53e2ad91b37c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6800', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6801', 'nonce': 3658047863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6802', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6803', 'nonce': 3658047863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6806', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6807', 'nonce': 3658047863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6804', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6805', 'nonce': 3658047863}]}, 'public_addr': '172.21.15.19:6801/3658047863', 'cluster_addr': '172.21.15.19:6803/3658047863', 'heartbeat_back_addr': '172.21.15.19:6807/3658047863', 'heartbeat_front_addr': '172.21.15.19:6805/3658047863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '5d7097a2-8165-4885-a356-f8e67851be5d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.870330810546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1988554297}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1988554297}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1988554297}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1988554297}]}, 'public_addr': '172.21.15.192:6817/1988554297', 'cluster_addr': '172.21.15.192:6819/1988554297', 'heartbeat_back_addr': '172.21.15.192:6823/1988554297', 'heartbeat_front_addr': '172.21.15.192:6821/1988554297', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 40, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 1071217311}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 1071217311}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 1071217311}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 1071217311}]}, 'public_addr': '172.21.15.192:6825/1071217311', 'cluster_addr': '172.21.15.192:6827/1071217311', 'heartbeat_back_addr': '172.21.15.192:6831/1071217311', 'heartbeat_front_addr': '172.21.15.192:6829/1071217311', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 1069316293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 1069316293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 1069316293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 1069316293}]}, 'public_addr': '172.21.15.192:6801/1069316293', 'cluster_addr': '172.21.15.192:6803/1069316293', 'heartbeat_back_addr': '172.21.15.192:6807/1069316293', 'heartbeat_front_addr': '172.21.15.192:6805/1069316293', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'fb77f072-c3ae-4914-81bd-473d30016320', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 3119270077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 3119270077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 3119270077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 3119270077}]}, 'public_addr': '172.21.15.192:6809/3119270077', 'cluster_addr': '172.21.15.192:6811/3119270077', 'heartbeat_back_addr': '172.21.15.192:6815/3119270077', 'heartbeat_front_addr': '172.21.15.192:6813/3119270077', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.795411+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.139237+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.562583+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.363254+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.344635+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.323050+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.381780+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.049941+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.2', 'osds': [6, 5]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T21:33:49.699 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 2.2 2024-09-15T21:33:49.699 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 2.2 2024-09-15T21:33:49.700 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:49.694+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:49.797 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:49.795+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:49.804 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:49.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 144.38365354499999, 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:49.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:49.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:49.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:49.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.811 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.360308699, 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.812 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:49.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:49.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:49.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:49.821 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.25894171600001, 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:49.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:49.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:49.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.258757898, 2024-09-15T21:33:49.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:49.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:49.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:49.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:49.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.836 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.25871905299999, 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:49.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:49.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.842 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.25866707899999, 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.843 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:49.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.258651351, 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.849 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:49.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:49.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.25863270299999, 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.856 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:49.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.860 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:49.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.25857475199999, 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:49.862 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:49.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.868 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.25855692100001, 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:49.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:49.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.874 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.258475895, 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.875 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:49.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:49.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.25837464, 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.881 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:49.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:49.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:49.887 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 143.258328011, 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:49.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:49.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.893 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 142.80098415800001, 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:49.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 142.800914968, 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.900 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:49.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:49.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 142.80064971199999, 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.906 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:49.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 142.800575442, 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.912 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:49.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:49.918 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 142.80054654099999, 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:49.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:49.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.924 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 142.800531213, 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:49.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:49.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:49.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.930 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 142.80044692499999, 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:49.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:49.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:49.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:49.936 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.937 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.937 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:49.937 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:49.937 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:49.938 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:33:49.939 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '23 of 30' 2024-09-15T21:33:49.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:49.940 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:49.935+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:50.039 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:50.037+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:50.090 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:50.090 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:50.090 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:50.099 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:50.140 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:50.139+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:50.206 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:50.207 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:50.207 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:50.207 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:50.207 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:50.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:50.260 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:50.257+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.881668899, 2024-09-15T21:33:50.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:50.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:50.332 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.880755867, 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:50.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:50.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.774887595, 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:50.337 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:50.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:50.341 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.774475325, 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:50.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:50.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.346 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.77402715599999, 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:50.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:50.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.351 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.77318880799999, 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:50.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:50.354 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.355 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.356 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.77227234200001, 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:50.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:50.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.32184477199999, 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:50.361 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:50.365 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.32139801299999, 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:50.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:50.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.370 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.32097077099999, 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.371 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:50.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:50.375 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.32050382200001, 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.376 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:50.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:50.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.320135279, 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.380 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.381 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.319718209, 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.385 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:50.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.31706719300001, 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:50.389 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:50.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.393 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.316369864, 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:50.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:50.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.03304496999999, 2024-09-15T21:33:50.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:50.400 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.402 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.02905504, 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.403 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:50.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:50.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.407 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.028398778, 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:50.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.410 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.02394522, 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.412 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:50.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.02121246499999, 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:50.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:50.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:50.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:50.421 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:50.423 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:50.423 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:50.419+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:50.475 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:50.550 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:50.547+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:50.572 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:50.572 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:50.573 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:50.573 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:50.582 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:50.624 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:50.621+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:50.632 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:50.633 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:50.649 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:50.649 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:50.666 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:50.666 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:50.682 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:50.682 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:50.699 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:50.700 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:50.717 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:50.717 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:50.725 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:50.722+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:50.732 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:50.732 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:50.734 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:50.734 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:50.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:50.751 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:50.751 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:50.826 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:50.823+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:50.856 INFO:teuthology.orchestra.run.smithi019.stderr:clear 2.2 pg_upmap mapping 2024-09-15T21:33:50.858 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '24 of 30' 2024-09-15T21:33:50.863 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:50.864 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:50.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:50.958 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:50.955+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:51.023 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:51.023 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:51.029 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:51.027+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:51.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:51.129 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:51.128+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:51.180 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:51.180 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:51.189 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:51.230 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:51.228+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:51.295 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:51.296 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:51.304 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:51.348 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:51.345+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:51.412 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:51.412 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:51.420 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:51.461 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:51.459+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:51.533 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:51.531+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:51.568 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:51.568 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:51.576 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:51.633 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:51.631+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:51.683 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:51.684 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:51.692 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:51.737 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:51.735+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:51.807 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:51.807 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:51.815 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:51.835 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:51.833+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:51.859 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '25 of 30' 2024-09-15T21:33:51.941 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:33:51.943 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:51.940+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:51.961 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:51.961 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:51.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:52.037 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:52.035+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:52.073 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:52.073 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:52.081 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:52.166 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:52.164+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:52.225 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:52.225 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:52.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:52.275 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:52.273+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:52.340 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:52.338+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:52.378 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:52.378 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:52.387 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:52.472 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:52.470+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:52.531 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:52.531 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:52.540 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:52.541 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:52.539+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:52.642 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:52.639+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:52.684 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:52.684 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:52.692 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:52.743 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:52.741+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:52.763 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:52.796 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:52.796 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:52.805 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:52.844 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:52.841+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:52.854 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:33:52.868 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '26 of 30' 2024-09-15T21:33:52.895 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:52.895 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:52.910 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:52.910 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:52.913 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:52.913 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:52.919 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:52.946 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:52.942+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:53.025 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:53.025 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:53.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:53.045 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:53.043+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:53.132 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:53.132 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:53.136 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:53.136 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:53.145 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:53.147 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:53.145+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:53.148 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:53.149 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:53.165 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:53.166 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:53.182 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:53.183 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:53.199 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:53.199 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:53.216 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:53.216 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:53.248 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:53.246+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.83206788999999, 2024-09-15T21:33:53.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:33:53.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.808723044, 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:33:53.261 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:33:53.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:33:53.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.267 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:33:53.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:33:53.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.70735606100001, 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.269 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:33:53.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:33:53.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.275 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.707172243, 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.276 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:33:53.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.279 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:33:53.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.282 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.707133398, 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:33:53.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.285 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:33:53.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.288 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.70708142399999, 2024-09-15T21:33:53.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:33:53.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:33:53.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.707065696, 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.296 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:33:53.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:33:53.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:33:53.302 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.70704704799999, 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.303 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:33:53.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:33:53.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:33:53.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.310 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.70698909699999, 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:33:53.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:33:53.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:33:53.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:33:53.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:33:53.317 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.70697126600001, 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.318 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:53.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:33:53.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.70689024000001, 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.324 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:33:53.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:33:53.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.706788985, 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.330 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:33:53.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:33:53.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.335 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:33:53.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.337 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.70674235600001, 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:33:53.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:33:53.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:33:53.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:33:53.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.24939850300001, 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.345 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:33:53.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.350 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.249329313, 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.351 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:33:53.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.356 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.249064057, 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.357 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:33:53.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.362 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.248989787, 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.363 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:53.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:33:53.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:33:53.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.24896088599999, 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.369 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:33:53.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.248945558, 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.375 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:33:53.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:33:53.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:33:53.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.24886126999999, 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:53.382 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:53.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:33:53.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:53.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:33:53.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:53.388 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:53.390 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:53.388+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:53.390 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:53.491 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:53.489+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:53.498 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:53.499 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:53.499 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:53.499 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:53.507 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:53.592 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:53.590+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:53.614 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:53.614 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:53.614 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:53.614 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:53.615 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:53.623 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:53.693 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:53.690+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:53.731 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 147.28806265, 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.732 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:33:53.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:33:53.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 147.287149618, 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:33:53.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.741 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 147.18128134599999, 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:33:53.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:33:53.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.745 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 147.18086907599999, 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.746 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.747 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.747 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:53.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:33:53.748 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.750 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.751 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 147.18042090700001, 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.752 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.753 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:33:53.754 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:33:53.755 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:53.756 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 147.17958255900001, 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:53.757 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.758 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 147.178666093, 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.761 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:33:53.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.72823852299999, 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:53.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.767 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:33:53.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:33:53.770 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.72779176399999, 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.771 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:33:53.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:33:53.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.72736452199999, 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:33:53.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:33:53.779 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.726897573, 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.780 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:33:53.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.783 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.784 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:33:53.785 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:53.785 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.72652902999999, 2024-09-15T21:33:53.785 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:33:53.785 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.785 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.785 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.785 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:33:53.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.789 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.72611196, 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.790 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.72346094400001, 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.794 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:33:53.796 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:33:53.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:33:53.798 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:33:53.799 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.72276361499999, 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.800 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:33:53.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.803 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.43943872099999, 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:33:53.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:33:53.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.435448791, 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:33:53.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:33:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:33:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:33:53.813 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:33:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.814 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.43479252899999, 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.815 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:33:53.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:33:53.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:33:53.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.430338971, 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.820 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:33:53.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.42760621599999, 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:53.825 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.826 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:53.827 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:33:53.827 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:33:53.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.827 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.827 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:53.827 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:53.828 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.829 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:53.830 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:53.832 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:53.832 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:53.830+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:53.864 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '27 of 30' 2024-09-15T21:33:53.933 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:53.930+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:54.015 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:54.016 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:54.016 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:54.016 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:54.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:54.070 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:54.067+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:54.134 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:54.134 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:54.134 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:54.134 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:54.135 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:54.135 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:54.131+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:54.143 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:54.235 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:54.232+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:54.290 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 141.83184071700001, 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:54.291 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:54.292 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.293 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:33:54.294 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.295 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:33:54.296 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 140.82595932500001, 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.297 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:33:54.298 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.299 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:54.300 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:33:54.300 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:33:54.300 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.300 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.300 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.301 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.302 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 1.3074399290000001, 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:54.303 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.304 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:33:54.305 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:33:54.306 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:54.307 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.308 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.309 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:54.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:54.336 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:54.334+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:54.437 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:54.435+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:54.457 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:54.458 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:54.458 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:54.458 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:54.466 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:54.538 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:54.536+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:54.573 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:54.573 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:54.573 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:54.581 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:54.640 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:54.638+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 143.27254139999999, 2024-09-15T21:33:54.730 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:54.731 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.732 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:54.733 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:54.733 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:54.733 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.733 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.733 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.734 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:54.735 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:54.736 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 57.610151002999999, 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.737 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:54.738 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:54.739 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.740 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 43.240117093999999, 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:54.741 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.742 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.743 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:54.744 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 37.511629601999999, 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:54.745 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.746 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:33:54.747 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:54.748 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:54.749 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:54.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:54.752 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:54.748+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:54.852 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:54.849+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:54.869 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '28 of 30' 2024-09-15T21:33:54.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:33:54.900 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:54.900 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:54.900 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:54.900 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:54.909 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:54.953 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:54.950+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:55.053 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:55.050+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:55.062 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:55.062 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:55.062 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:55.062 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:55.063 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:55.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:55.154 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:55.152+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:55.221 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:55.221 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:55.221 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:55.221 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:55.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.221 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 58.102085510000002, 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:55.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:55.223 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.224 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:55.225 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:55.226 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:55.226 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:55.226 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:55.226 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:55.226 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:55.226 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:55.226 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:55.227 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:55.230 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:55.271 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:55.268+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:55.345 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:55.346 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:55.346 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:55.346 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:55.351 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:55.351 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:55.355 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:55.355 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:55.353+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:55.369 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:55.369 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:55.385 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:55.385 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:55.401 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:55.401 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:55.417 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:55.418 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:55.434 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:55.435 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:55.451 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:55.451 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:55.456 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:55.454+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:55.468 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:55.468 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:55.500 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:55.501 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:55.509 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:55.593 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:55.591+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:55.652 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:55.653 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:55.658 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:55.656+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:55.661 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:55.759 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:55.757+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:55.805 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:55.805 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:55.814 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:55.860 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:55.858+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:55.862 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:33:55.871 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:33:55.871 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:33:55.871 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T21:33:55.871 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T21:33:55.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '29 of 30' 2024-09-15T21:33:55.918 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:55.918 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:55.926 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:55.962 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:55.960+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:56.029 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:56.029 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:56.038 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:56.061 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:56.059+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:56.144 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:56.144 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:56.150 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:56.163 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:56.161+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:56.183 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:33:56.183 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:56.183 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 54, 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2024-09-15T21:31:12.329884+0000", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2024-09-15T21:33:55.853458+0000", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "last_up_change": "2024-09-15T21:31:25.414443+0000", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "last_in_change": "2024-09-15T21:33:28.789828+0000", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_num": 5799936, 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_set": [ 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "pglog_hardlimit", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "purged_snapdirs", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_deletes", 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "sortbitwise" 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:56.184 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_version": 5, 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_snapshot": "", 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_max": 2, 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd": 8, 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "require_min_compat_client": "luminous", 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "min_compat_client": "jewel", 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "require_osd_release": "squid", 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_crimson": false, 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "pools": [ 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.185 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 1, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": ".mgr", 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:22.622713+0000", 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool", 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:33:56.186 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 1, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 1, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 1, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 1, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 1, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:33:56.187 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "18", 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 0, 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 0, 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:33:56.188 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:33:56.189 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:33:56.190 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:33:56.192 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:33:56.192 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:33:56.192 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:33:56.192 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:33:56.192 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:33:56.192 INFO:teuthology.orchestra.run.smithi019.stdout: "options": { 2024-09-15T21:33:56.192 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_max": 32, 2024-09-15T21:33:56.192 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_min": 1 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": {} 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 8, 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 8, 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 0.25, 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 2, 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 2, 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 0.98379135131835938, 2024-09-15T21:33:56.193 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 2, 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": "rbd", 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:30.313202+0000", 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 8193, 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:33:56.194 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 8, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 8, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 8, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 8, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 8, 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:33:56.195 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "22", 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 2, 2024-09-15T21:33:56.196 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 22, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:33:56.197 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:33:56.198 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "options": {}, 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd": {} 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 1.9900000095367432, 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 1.9900000095367432, 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 0.87999999523162842, 2024-09-15T21:33:56.199 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 1.75, 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 1.75, 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 0.98379135131835938, 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 0.98000001907348633 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "osds": [ 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "f7eb9035-e8c2-4d90-a198-2bed78d5a3b5", 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:56.200 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 27, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:56.201 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.202 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.202 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.202 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6824", 2024-09-15T21:33:56.203 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:56.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.203 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.203 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6825", 2024-09-15T21:33:56.204 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:56.204 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.204 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.204 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:56.204 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6826", 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6827", 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.206 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:56.206 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.207 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6830", 2024-09-15T21:33:56.207 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:56.207 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.207 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6831", 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6828", 2024-09-15T21:33:56.208 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:56.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.209 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.209 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.209 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6829", 2024-09-15T21:33:56.209 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:33:56.209 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.210 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.210 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.210 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6825/368705522", 2024-09-15T21:33:56.210 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6827/368705522", 2024-09-15T21:33:56.210 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6831/368705522", 2024-09-15T21:33:56.210 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6829/368705522", 2024-09-15T21:33:56.210 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a", 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:56.211 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 36, 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6816", 2024-09-15T21:33:56.212 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6817", 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.213 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6818", 2024-09-15T21:33:56.214 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:56.214 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.214 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.214 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6819", 2024-09-15T21:33:56.214 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:56.215 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.215 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.215 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:56.215 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6822", 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6823", 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:56.216 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6820", 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6821", 2024-09-15T21:33:56.217 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6817/3888640163", 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6819/3888640163", 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6823/3888640163", 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6821/3888640163", 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:56.218 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "180bae67-0f64-45a0-9a58-f4b8ff2b90a5", 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 36, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:56.219 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:56.220 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:56.220 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.220 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.220 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6808", 2024-09-15T21:33:56.220 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:56.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.220 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6809", 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6810", 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:56.221 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6811", 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6814", 2024-09-15T21:33:56.223 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:56.223 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.223 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.223 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.223 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6815", 2024-09-15T21:33:56.223 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6812", 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:56.224 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6813", 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6809/1831747042", 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6811/1831747042", 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6815/1831747042", 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6813/1831747042", 2024-09-15T21:33:56.225 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:56.226 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:56.226 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:56.226 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.226 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:33:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d05cec44-e228-4297-98c8-53e2ad91b37c", 2024-09-15T21:33:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:56.227 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:56.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.229 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6802", 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6803", 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6806", 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6807", 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:56.231 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6804", 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6805", 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:33:56.232 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6801/3658047863", 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6803/3658047863", 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6807/3658047863", 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6805/3658047863", 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "5d7097a2-8165-4885-a356-f8e67851be5d", 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 0.870330810546875, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:56.234 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6816", 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6817", 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6818", 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6819", 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.236 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6822", 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.237 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6823", 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.238 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6820", 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6821", 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6817/1988554297", 2024-09-15T21:33:56.239 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6819/1988554297", 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1988554297", 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1988554297", 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.240 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f", 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 14, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 52, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:56.241 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6824", 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6825", 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.242 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6826", 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6827", 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6830", 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6831", 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.244 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.245 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:56.245 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.245 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6828", 2024-09-15T21:33:56.245 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:56.245 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.246 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6829", 2024-09-15T21:33:56.246 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:33:56.246 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.246 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6825/1071217311", 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6827/1071217311", 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6831/1071217311", 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6829/1071217311", 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612", 2024-09-15T21:33:56.247 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 43, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6800", 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6801", 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6802", 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6803", 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.250 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6806", 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6807", 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:56.251 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6804", 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6805", 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.252 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6801/1069316293", 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6803/1069316293", 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6807/1069316293", 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6805/1069316293", 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "fb77f072-c3ae-4914-81bd-473d30016320", 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:33:56.253 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 43, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6808", 2024-09-15T21:33:56.254 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6809", 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.255 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6810", 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6811", 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.256 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6814", 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6815", 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:33:56.257 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6812", 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6813", 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6809/3119270077", 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6811/3119270077", 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6815/3119270077", 2024-09-15T21:33:56.258 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6813/3119270077", 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_xinfo": [ 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:56.259 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.795411+0000", 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.139237+0000", 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.562583+0000", 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:56.261 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.363254+0000", 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:56.262 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.344635+0000", 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.323050+0000", 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.263 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.381780+0000", 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:33:56.264 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.049941+0000", 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap": [], 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_items": [], 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_primaries": [], 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_temp": [], 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_temp": [], 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklist": {}, 2024-09-15T21:33:56.265 INFO:teuthology.orchestra.run.smithi019.stdout: "range_blocklist": {}, 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profiles": { 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "default": { 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-failure-domain": "osd", 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "k": "2", 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "m": "1", 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "plugin": "jerasure", 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "technique": "reed_sol_van" 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps_queue": [], 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "new_removed_snaps": [], 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "new_purged_snaps": [], 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_node_flags": {}, 2024-09-15T21:33:56.266 INFO:teuthology.orchestra.run.smithi019.stdout: "device_class_flags": {}, 2024-09-15T21:33:56.267 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": { 2024-09-15T21:33:56.267 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_enabled": false, 2024-09-15T21:33:56.267 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_bucket_count": 0, 2024-09-15T21:33:56.267 INFO:teuthology.orchestra.run.smithi019.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:33:56.267 INFO:teuthology.orchestra.run.smithi019.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:33:56.267 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_bucket": 0 2024-09-15T21:33:56.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:56.267 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:56.269 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:56.269 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:56.270 INFO:tasks.thrashosds.thrasher:j is {'epoch': 54, 'fsid': '13019d66-d4b6-4e2a-8322-38bcb4210bcf', 'created': '2024-09-15T21:31:12.329884+0000', 'modified': '2024-09-15T21:33:55.853458+0000', 'last_up_change': '2024-09-15T21:31:25.414443+0000', 'last_in_change': '2024-09-15T21:33:28.789828+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T21:31:22.622713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9837913513183594, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:31:30.313202+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9837913513183594, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.9800000190734863}}], 'osds': [{'osd': 0, 'uuid': 'f7eb9035-e8c2-4d90-a198-2bed78d5a3b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6824', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6825', 'nonce': 368705522}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6826', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6827', 'nonce': 368705522}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6830', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6831', 'nonce': 368705522}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6828', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6829', 'nonce': 368705522}]}, 'public_addr': '172.21.15.19:6825/368705522', 'cluster_addr': '172.21.15.19:6827/368705522', 'heartbeat_back_addr': '172.21.15.19:6831/368705522', 'heartbeat_front_addr': '172.21.15.19:6829/368705522', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd2bcecd4-0bb0-4b38-b395-8b6dbb48f53a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6816', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6817', 'nonce': 3888640163}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6818', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6819', 'nonce': 3888640163}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6822', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6823', 'nonce': 3888640163}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6820', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6821', 'nonce': 3888640163}]}, 'public_addr': '172.21.15.19:6817/3888640163', 'cluster_addr': '172.21.15.19:6819/3888640163', 'heartbeat_back_addr': '172.21.15.19:6823/3888640163', 'heartbeat_front_addr': '172.21.15.19:6821/3888640163', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '180bae67-0f64-45a0-9a58-f4b8ff2b90a5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6808', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6809', 'nonce': 1831747042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6810', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6811', 'nonce': 1831747042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6814', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6815', 'nonce': 1831747042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6812', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6813', 'nonce': 1831747042}]}, 'public_addr': '172.21.15.19:6809/1831747042', 'cluster_addr': '172.21.15.19:6811/1831747042', 'heartbeat_back_addr': '172.21.15.19:6815/1831747042', 'heartbeat_front_addr': '172.21.15.19:6813/1831747042', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd05cec44-e228-4297-98c8-53e2ad91b37c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6800', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6801', 'nonce': 3658047863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6802', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6803', 'nonce': 3658047863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6806', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6807', 'nonce': 3658047863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6804', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6805', 'nonce': 3658047863}]}, 'public_addr': '172.21.15.19:6801/3658047863', 'cluster_addr': '172.21.15.19:6803/3658047863', 'heartbeat_back_addr': '172.21.15.19:6807/3658047863', 'heartbeat_front_addr': '172.21.15.19:6805/3658047863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '5d7097a2-8165-4885-a356-f8e67851be5d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.870330810546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1988554297}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1988554297}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1988554297}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1988554297}]}, 'public_addr': '172.21.15.192:6817/1988554297', 'cluster_addr': '172.21.15.192:6819/1988554297', 'heartbeat_back_addr': '172.21.15.192:6823/1988554297', 'heartbeat_front_addr': '172.21.15.192:6821/1988554297', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 1071217311}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 1071217311}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 1071217311}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 1071217311}]}, 'public_addr': '172.21.15.192:6825/1071217311', 'cluster_addr': '172.21.15.192:6827/1071217311', 'heartbeat_back_addr': '172.21.15.192:6831/1071217311', 'heartbeat_front_addr': '172.21.15.192:6829/1071217311', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 1069316293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 1069316293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 1069316293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 1069316293}]}, 'public_addr': '172.21.15.192:6801/1069316293', 'cluster_addr': '172.21.15.192:6803/1069316293', 'heartbeat_back_addr': '172.21.15.192:6807/1069316293', 'heartbeat_front_addr': '172.21.15.192:6805/1069316293', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'fb77f072-c3ae-4914-81bd-473d30016320', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 3119270077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 3119270077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 3119270077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 3119270077}]}, 'public_addr': '172.21.15.192:6809/3119270077', 'cluster_addr': '172.21.15.192:6811/3119270077', 'heartbeat_back_addr': '172.21.15.192:6815/3119270077', 'heartbeat_front_addr': '172.21.15.192:6813/3119270077', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.795411+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.139237+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.562583+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.363254+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.344635+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.323050+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.381780+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.049941+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T21:33:56.270 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-15T21:33:56.270 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:56.266+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:56.270 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:56.369 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:56.367+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:56.374 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:56.374 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:56.383 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:56.470 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:56.468+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:56.525 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:56.525 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:56.535 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:56.571 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:56.569+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:56.639 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:56.639 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:56.648 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:56.672 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:56.670+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:56.751 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:56.751 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:56.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:33:56.772 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:56.771+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:56.874 INFO:tasks.mon_recovery:adding mon b back in 2024-09-15T21:33:56.874 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:33:56.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:33:56.874 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:56.871+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:56.875 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:56.875 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:56.876 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:33:56.876 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-09-15T21:33:56.876 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:33:56.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:33:56.975 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:56.972+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:57.035 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:57.035 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:57.045 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:33:57.075 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:57.074+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:57.152 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:57.152 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:57.165 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:33:57.208 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:57.206+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:57.251 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":50,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":29,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:33:57.273 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:57.273 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:57.276 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:57.274+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:57.281 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:33:57.378 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:57.376+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:57.430 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:57.439 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:33:57.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:33:57.483 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:57.481+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:57.548 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:33:57.548 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:33:57.557 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:33:57.580 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:57.578+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:57.607 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:57.608 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:57.625 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:57.625 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:57.641 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:57.642 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:57.658 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:57.658 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:57.667 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:57.667 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:57.667 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:57.667 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:57.675 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:57.675 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:57.676 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:33:57.692 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:57.692 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:57.693 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:57.690+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:57.709 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:57.710 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:33:57.745 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:33:57.745 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:33:57.796 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:57.794+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:57.796 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:57.796 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:57.796 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:57.797 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:57.797 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:57.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:33:57.879 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:33:57.889 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:57.887+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:57.954 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:57.954 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:57.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:57.954 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:57.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.954 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 146.49667612799999, 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:57.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:33:57.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:33:57.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:33:57.959 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 60.834285731000001, 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:33:57.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.964 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:33:57.964 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:57.964 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 46.464251822000001, 2024-09-15T21:33:57.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:33:57.964 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:57.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:33:57.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:57.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:57.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.968 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 40.735764330000002, 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:57.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:57.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:57.974 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:57.976 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:33:57.984 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:33:57.982+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:33:58.084 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:58.082+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:58.085 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:58.085 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:58.085 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:33:58.085 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:58.094 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:33:58.185 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:58.183+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:58.242 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:58.242 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:33:58.242 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:33:58.242 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:33:58.242 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:58.251 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:33:58.295 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:58.293+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:58.358 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:33:58.358 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:33:58.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:33:58.358 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:33:58.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.358 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 61.239116291000002, 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:33:58.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:33:58.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:58.362 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:33:58.363 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:33:58.363 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:33:58.367 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:33:58.411 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:58.409+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:58.480 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:58.481 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:58.481 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:58.487 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:58.485+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:58.489 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:33:58.588 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:58.585+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:58.635 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:58.635 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:58.635 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:58.643 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:33:58.727 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:58.725+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:58.786 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:58.786 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:58.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:58.786 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:33:58.787 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:58.790 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:58.788+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:58.795 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:33:58.865 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:33:58.921 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:33:58.919+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:33:58.938 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:58.939 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:58.939 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:58.947 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:33:58.991 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:58.989+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:59.049 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:59.050 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:59.050 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:59.050 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:59.050 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:59.058 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:33:59.091 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:33:59.089+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:33:59.161 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:59.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.706604067, 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:33:59.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:33:59.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:33:59.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:59.168 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.70046443699999, 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:59.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:59.172 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.69729626099999, 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.174 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.175 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.176 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.69675050800001, 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:33:59.178 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:33:59.179 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.180 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.7041122, 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.181 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:33:59.182 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.183 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.184 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.185 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:33:59.186 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 146.70081937200001, 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:33:59.187 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.188 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:33:59.189 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:33:59.190 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:59.191 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.69826860200001, 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:59.192 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.193 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.194 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.195 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:33:59.196 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:33:59.197 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 145.694867913, 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.198 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:33:59.199 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.200 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.201 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.670632775000001, 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.202 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.203 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.204 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.205 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 47.667626609999999, 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:33:59.206 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.207 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.208 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:33:59.209 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.210 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:59.212 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:33:59.212 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:33:59.210+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:33:59.312 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:33:59.310+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:59.354 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:59.363 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:33:59.412 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:59.410+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:59.466 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:59.466 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:59.466 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:59.466 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:59.466 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:59.475 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:33:59.516 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:59.514+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:59.615 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:59.613+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:59.620 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:59.620 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:33:59.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:33:59.620 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:33:59.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 96.703437624000003, 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:33:59.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:33:59.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:59.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.677100205000002, 2024-09-15T21:33:59.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:33:59.626 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.626 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.626 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:59.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:59.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:33:59.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:33:59.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 88.673666936999993, 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:59.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:33:59.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:33:59.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 48.123065357000002, 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.635 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:33:59.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:33:59.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 36.703482733000001, 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:33:59.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:33:59.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:33:59.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:33:59.644 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:33:59.644 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:59.645 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:33:59.716 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:33:59.714+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:33:59.748 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:59.748 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:59.748 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:33:59.748 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:59.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:33:59.757 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:33:59.841 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:33:59.839+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:33:59.894 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:33:59.894 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:33:59.901 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:33:59.901 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:33:59.901 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:33:59.901 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:33:59.901 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:33:59.910 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:33:59.911 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:33:59.912 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:33:59.917 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:33:59.915+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:33:59.928 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:33:59.928 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:33:59.944 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:33:59.945 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:33:59.962 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:33:59.962 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:33:59.979 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:33:59.979 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:33:59.996 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:33:59.997 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:00.013 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:00.013 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:00.018 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:00.016+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:00.054 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:00.054 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:00.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:00.119 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:00.117+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:00.172 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:00.172 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:00.180 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:00.225 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:00.223+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:00.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:00.289 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:00.289 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:00.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:00.320 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:00.319+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:00.421 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:00.419+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:00.446 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:00.446 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:00.456 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:00.522 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:00.520+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:00.563 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:00.563 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:00.572 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:00.624 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:00.622+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:00.679 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:00.679 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:00.688 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:00.731 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:00.729+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:00.795 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:00.795 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:00.805 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:00.846 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:00.844+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:00.880 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:34:00.927 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:00.925+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:00.954 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:00.954 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:00.963 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:01.028 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:01.026+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:01.071 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:01.071 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:01.079 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:01.129 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:01.128+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:01.187 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:01.187 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:01.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:01.230 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:01.229+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:01.270 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:01.271 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:01.331 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:01.329+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:01.343 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:01.352 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:01.432 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:01.431+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:01.459 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:01.460 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:01.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:01.553 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:01.551+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:01.617 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:01.618 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:01.626 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:01.671 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:01.669+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:01.735 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:01.733+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:01.741 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:01.741 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:01.751 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:01.836 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:01.834+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:01.896 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:01.896 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:01.905 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:01.946 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:01.944+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:02.025 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:02.048 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:02.048 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:02.056 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:02.070 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:02.068+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:02.138 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:02.136+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:02.159 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:02.159 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:02.168 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:02.240 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:02.238+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:02.311 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:02.311 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:02.320 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:02.341 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:02.338+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:02.423 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:02.424 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:02.432 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:02.441 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:02.439+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:02.535 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:02.535 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:02.542 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:02.540+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:02.544 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:02.643 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:02.641+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:02.646 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:02.646 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:02.655 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:02.744 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:02.742+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:02.759 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:02.759 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:02.768 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:02.845 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:02.843+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:02.911 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:02.911 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:02.920 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:02.945 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:02.944+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:03.022 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:03.023 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:03.031 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:03.076 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:03.074+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:03.135 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:03.135 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:03.144 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:03.147 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:03.145+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:03.248 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:03.246+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:03.265 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:03.265 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:03.275 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:03.359 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:03.357+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:03.425 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:03.425 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:03.434 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:03.448 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:03.447+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:03.549 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:03.547+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:03.581 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:03.582 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:03.591 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:03.650 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:03.648+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:03.698 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:03.698 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:03.708 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:03.753 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:03.751+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:03.816 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:03.816 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:03.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:03.866 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:03.864+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:03.952 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:03.951+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:03.972 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:03.973 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:03.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:04.066 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:04.064+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:04.130 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:04.130 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:04.138 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:04.154 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:04.152+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:04.254 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:04.253+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:04.286 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:04.286 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:04.295 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:04.356 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:04.354+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:04.402 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:04.402 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:04.411 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:04.457 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:04.455+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:04.558 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:04.556+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:04.559 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:04.559 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:04.567 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:04.658 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:04.656+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:04.674 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:04.674 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:04.683 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:04.771 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:04.769+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:04.832 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:04.832 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:04.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:04.883 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:04.881+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:04.957 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:04.958 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:04.962 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:04.960+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:04.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:05.024 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:05.024 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:05.042 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:05.043 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:05.059 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:05.059 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:05.062 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:05.061+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:05.071 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:05.072 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:05.075 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:05.076 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:05.080 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:05.092 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:05.093 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:05.109 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:05.109 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:05.126 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:05.126 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:05.142 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:05.143 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:05.163 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:05.161+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:05.184 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:05.184 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:05.185 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:05.185 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:34:05.185 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:05.193 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:05.264 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:05.262+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:05.268 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:05.281 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-09-15T21:34:05.281 INFO:tasks.mon_recovery:removing mon c 2024-09-15T21:34:05.281 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:34:05.282 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:05.335 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:34:05.335 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-09-15T21:34:05.336 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:05.339 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:05.339 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:05.340 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:05.349 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:05.377 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:05.375+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:05.451 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:05.451 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:05.451 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:05.460 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:05.505 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:05.503+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:05.565 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:05.565 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:05.566 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 154.110080393, 2024-09-15T21:34:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:05.567 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:05.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.574 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.575 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.576 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:05.577 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:05.578 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:05.579 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:05.580 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.581 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:05.582 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 154.103940763, 2024-09-15T21:34:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.583 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.584 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:05.585 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:05.586 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.587 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.588 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.590 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.591 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 154.10077258699999, 2024-09-15T21:34:05.592 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:05.593 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.594 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.595 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:05.596 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:05.597 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:05.598 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.599 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.601 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 154.10022683400001, 2024-09-15T21:34:05.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:05.603 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:05.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:05.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:05.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:05.610 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 153.107588526, 2024-09-15T21:34:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:05.611 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:05.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:05.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:05.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 153.10429569799999, 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.618 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:05.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:05.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 152.10174492799999, 2024-09-15T21:34:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:05.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:05.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:05.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 152.098344239, 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:05.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.074109100999998, 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.634 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:05.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.071102936000003, 2024-09-15T21:34:05.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:05.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:05.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.642 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:05.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:05.643 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:05.644 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:05.645 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:05.642+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:05.745 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:05.743+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:05.746 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:05.787 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:05.787 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:05.787 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:05.787 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:05.795 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:05.846 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:05.844+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:05.897 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:05.898 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:05.898 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:05.906 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:05.950 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:05.948+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 103.09531929000001, 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.010 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:06.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.014 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.068981871000005, 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:06.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.065548602999996, 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:06.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:06.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.023 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.023 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:06.023 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 54.514947022999998, 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.027 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 43.095364398999997, 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:06.028 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:06.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:06.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:06.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.032 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.032 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.032 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.032 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:06.033 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:06.048 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:06.046+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:06.149 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:06.147+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:06.177 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:06.177 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:06.177 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:06.177 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:06.186 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:06.249 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:06.247+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:06.271 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:06.272 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:06.329 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:06.329 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:06.329 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:06.330 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:06.330 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:06.338 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:06.382 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:06.380+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:06.442 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 161.02175683499999, 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:06.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:06.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:06.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:06.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.998411989, 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:06.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.89704500600001, 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.458 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:06.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:06.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.896861188, 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.464 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:06.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:06.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.896822343, 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:06.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:06.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.896770369, 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.477 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:06.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.896754641, 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:06.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:06.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:06.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.89673599299999, 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.489 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:06.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.89667804199999, 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.495 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:06.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:06.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.89666021100001, 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:06.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:06.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:06.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.89657918500001, 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.508 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:06.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:06.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.89647793, 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.514 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:06.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.89643130100001, 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.520 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:06.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:06.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.525 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.526 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.43908744800001, 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.527 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.528 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:06.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:06.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.439018258, 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:06.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.534 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:06.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.438753002, 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.539 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:06.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:06.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.545 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.438678732, 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.546 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:06.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:06.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.43864983099999, 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.552 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:06.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:06.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.438634503, 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.558 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:06.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.561 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.562 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:06.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.43855021499999, 2024-09-15T21:34:06.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:06.565 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:06.565 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:06.565 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:06.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:06.565 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:06.568 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:06.568 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.568 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:06.568 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.568 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:06.568 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:06.568 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.569 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:06.570 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:06.571 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.572 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:06.573 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:06.575 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:06.573+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:06.575 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:06.676 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:06.674+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:06.691 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:06.691 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:06.691 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:06.700 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:06.776 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:06.775+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:06.807 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:06.807 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:06.816 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:06.901 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:06.899+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:06.966 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:06.966 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 160.522265339, 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:06.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:06.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:06.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 160.521352307, 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:06.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:06.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:06.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 160.41548403499999, 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.976 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:06.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:06.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 160.41507176499999, 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.981 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:06.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 160.41462359600001, 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:06.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:06.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.990 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 160.41378524800001, 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:06.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 160.412868782, 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:06.995 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:06.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:06.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:06.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:07.000 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.96244121199999, 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:07.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:07.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.006 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.96199445299999, 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:07.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:07.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:07.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:07.011 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.96156721099999, 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:07.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:07.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.016 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.961100262, 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:07.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:07.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.96073171899999, 2024-09-15T21:34:07.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:07.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.960314649, 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:07.026 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.95766363300001, 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:07.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.034 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.035 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.95696630399999, 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.036 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:07.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.67364140999999, 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.041 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.044 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:07.045 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.66965148, 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.046 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:07.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.047 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:07.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.050 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.66899521799999, 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.051 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.052 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:07.053 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.054 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.055 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.66454166, 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.056 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:07.057 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:07.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.66180890499999, 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:07.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.063 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:07.064 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:07.065 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:07.067 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:07.067 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:07.064+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:07.156 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:07.168 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:07.166+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:07.216 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:07.217 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:07.217 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:07.217 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:07.226 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:07.270 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:07.268+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:07.283 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:07.283 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:07.300 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:07.301 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:07.317 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:07.317 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:07.333 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:07.333 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:07.335 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:07.335 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:07.344 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:07.351 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:07.351 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:07.368 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:07.369 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:07.369 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:07.368+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:07.389 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:07.389 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:07.403 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:07.404 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:07.459 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:07.459 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:07.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:07.470 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:07.468+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:07.572 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:07.570+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:07.577 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:07.577 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:07.586 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:07.673 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:07.671+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:07.693 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:07.702 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:07.773 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:07.771+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:07.811 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:07.811 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:07.819 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:07.905 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:07.903+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:07.968 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:07.968 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:07.975 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:07.973+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:07.977 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:08.076 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:08.074+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:08.124 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:08.124 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:08.132 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:08.217 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:08.215+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:08.277 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:08.276+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:08.280 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:08.280 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:08.289 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:08.379 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:08.377+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:08.404 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:08.404 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:08.413 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:08.480 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:08.478+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:08.557 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:08.557 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:08.566 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:08.610 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:08.608+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:08.670 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:08.670 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:08.679 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:08.720 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:08.718+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:08.761 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:08.782 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:08.780+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:08.823 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:08.823 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:08.833 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:08.917 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:08.915+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:08.975 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:08.975 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:08.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:08.984 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:08.981+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:09.084 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:09.082+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:09.128 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:09.128 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:09.137 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:09.140 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:09.185 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:09.183+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:09.239 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:09.239 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:09.248 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:09.289 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:09.287+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:09.388 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:09.386+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:09.391 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:09.391 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:09.400 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:09.416 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:09.488 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:09.486+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:09.503 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:09.512 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:09.542 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:09.542 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:09.558 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:09.559 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:09.575 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:09.575 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:09.589 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:09.587+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:09.591 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:09.591 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:09.607 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:09.608 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:09.625 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:09.626 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:09.642 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:09.642 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:09.659 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:09.659 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:09.659 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:09.659 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:09.659 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:09.660 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:09.668 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:09.712 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:09.710+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:09.770 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:09.771 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:09.771 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:09.779 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:09.820 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:09.818+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:09.892 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:09.890+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:09.924 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:09.924 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.50321672499999, 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.925 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:09.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:09.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:09.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.932 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.479871879, 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:09.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:09.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:09.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:09.939 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.37850489600001, 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:09.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:09.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:09.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.378321078, 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:09.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:09.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:09.951 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.37828223299999, 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.952 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.37823025899999, 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.958 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:09.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.378214531, 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:09.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:09.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.37819588299999, 2024-09-15T21:34:09.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:09.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:09.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.37813793199999, 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:09.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:09.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:09.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:09.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.37812010100001, 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:10.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:10.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.378039075, 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:10.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.055 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:10.056 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:10.057 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.37793782, 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.058 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:10.059 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.060 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.061 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:10.062 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:10.063 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.064 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.377891191, 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.065 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.066 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.067 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:10.068 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:10.069 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.070 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.92054733800001, 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.071 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.072 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.073 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.074 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.075 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.075 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:10.076 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:10.077 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.078 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.920478148, 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:10.079 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:10.080 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:10.081 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.082 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:10.083 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.084 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.920212892, 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.085 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:10.086 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.087 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:10.088 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.089 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:10.090 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.920138622, 2024-09-15T21:34:10.091 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.092 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.093 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.094 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:10.095 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.096 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.097 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.098 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:10.098 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.92010972099999, 2024-09-15T21:34:10.098 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:10.098 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.098 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.098 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.098 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.099 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.100 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.102 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:10.103 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.104 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.920094393, 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.105 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.106 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.107 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:10.108 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.109 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.110 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.92001010499999, 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:10.111 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.112 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:10.113 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.114 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:10.115 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:10.116 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:10.117 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:10.118 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:10.119 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:10.116+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:10.220 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:10.218+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:10.235 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:10.235 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:10.235 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:10.244 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:10.320 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:10.318+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:10.352 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:10.353 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:10.353 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:10.361 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:10.421 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:10.419+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 164.067215307, 2024-09-15T21:34:10.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:10.512 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:10.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 164.066302275, 2024-09-15T21:34:10.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:10.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:10.520 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.96043400299999, 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:10.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:10.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:10.525 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.96002173299999, 2024-09-15T21:34:10.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:10.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.95957356400001, 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:10.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:10.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.95873521600001, 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.541 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.95781875, 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.542 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:10.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.546 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.50739118000001, 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.547 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:10.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.551 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.50694442100001, 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.553 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:10.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.556 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.50651717900001, 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.557 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:10.558 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.561 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.50605023, 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.562 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:10.563 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.564 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.564 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.564 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:10.564 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:10.564 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.564 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.564 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:10.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:10.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.50568168699999, 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:10.568 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.570 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.571 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.571 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:10.571 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:10.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.571 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.571 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.571 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:10.571 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:10.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.572 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.572 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.572 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:10.572 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:10.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.573 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.573 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.573 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:10.573 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:10.573 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.573 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.573 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.50526461699999, 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.574 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:10.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.576 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:10.577 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.50261360100001, 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:10.579 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.580 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.581 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:10.581 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:10.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.581 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.581 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.581 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:10.581 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.582 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:10.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.50191627199999, 2024-09-15T21:34:10.584 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:10.585 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.585 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.585 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.585 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.585 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.585 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.585 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:10.585 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.586 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.587 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:10.588 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.589 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.21859137800001, 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.590 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.591 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.591 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.591 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.591 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:10.591 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.592 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.592 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.592 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:10.592 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.592 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.592 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.592 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.593 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:10.594 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.595 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.595 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.595 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:10.595 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:10.595 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.595 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.596 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.596 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:10.596 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:10.596 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.596 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.596 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.597 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:10.597 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:10.597 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.597 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.597 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.598 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.598 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.598 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:10.598 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:10.598 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.214601448, 2024-09-15T21:34:10.599 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:10.599 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.599 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.599 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.599 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.600 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.600 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.600 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:10.600 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.600 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.600 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:10.601 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.602 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.603 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.21394518599999, 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:10.604 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.605 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:10.606 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.607 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.608 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.20949162799999, 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.609 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.610 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.611 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:10.612 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.613 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.20675887300001, 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.614 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.615 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:10.616 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.617 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.618 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:10.620 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:10.620 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:10.618+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:10.720 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:10.718+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:10.729 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:10.729 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:10.729 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:10.729 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:10.737 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:10.822 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:10.820+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:10.845 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:10.846 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:10.846 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:10.846 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:10.846 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:10.855 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:10.923 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:10.921+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:10.963 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:10.963 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:10.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:10.963 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:10.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.963 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 158.504164162, 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:10.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:10.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:10.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:10.968 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 157.49828277, 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:10.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 17.979763374000001, 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:10.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:10.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:10.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.979 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:10.980 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:10.980 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:10.981 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:11.023 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:11.021+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:11.088 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:11.088 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:11.088 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:11.089 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:11.097 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:11.141 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:11.139+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:11.204 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:11.204 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:11.204 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:11.204 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:11.204 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:11.213 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:11.225 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:11.224+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:11.273 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:11.273 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:11.326 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:11.324+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:11.360 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:11.360 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:11.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:11.360 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:11.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 159.90294588699999, 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:11.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:11.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:11.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:11.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:11.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:11.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.365 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 74.240555490000006, 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:11.366 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:11.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:11.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 59.870521580999998, 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:11.370 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.371 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:11.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:11.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 54.142034088999999, 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:11.374 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:11.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:11.378 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:11.379 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:11.427 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:11.426+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:11.528 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:11.528 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:11.528 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:11.528 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:11.528 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:11.526+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:11.536 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:11.629 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:11.627+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:11.644 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:11.644 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:11.645 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:11.645 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:11.645 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:11.653 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:11.671 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:11.730 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:11.728+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:11.803 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 74.684497699999994, 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:11.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:11.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:34:11.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.808 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:11.808 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:34:11.808 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:34:11.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:11.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:11.808 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:11.809 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:34:11.809 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:34:11.809 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.809 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:11.809 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.809 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:11.809 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:11.809 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:11.813 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:11.857 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:11.855+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:11.929 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:11.929 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:11.929 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:11.929 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:11.932 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:11.930+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:11.938 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:12.033 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:12.031+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:12.081 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:12.082 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:12.082 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:12.090 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:12.134 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:12.132+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:12.153 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:12.234 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:12.234 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:34:12.234 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:12.235 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:12.232+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:12.243 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:12.335 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:12.333+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:12.387 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:12.387 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:12.387 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:12.387 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:12.396 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:12.435 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:12.434+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:12.499 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:12.499 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:12.499 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:12.507 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:12.532 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:12.537 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:12.535+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:12.611 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.611 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 161.156615018, 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:12.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:12.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:12.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:12.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.618 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 161.15047538799999, 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:12.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:12.622 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 161.14730721199999, 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:12.623 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:12.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:12.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:12.627 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 161.146761459, 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:12.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 160.15412315099999, 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:12.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:12.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:12.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.637 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:12.638 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 160.15083032300001, 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:12.639 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:12.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:12.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.14827955300001, 2024-09-15T21:34:12.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:12.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:12.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:12.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:12.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:12.650 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 159.14487886399999, 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:12.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.120643725999997, 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:12.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:12.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:12.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.117637561000002, 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:12.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:12.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:12.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:12.663 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:12.665 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:12.665 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:12.663+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:12.765 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:12.763+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:12.809 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:12.809 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:12.809 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:12.817 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:12.866 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:12.864+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:12.920 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:12.920 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:12.920 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:12.928 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:12.966 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:12.965+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:13.031 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:13.031 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 110.11710628599999, 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:13.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.035 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.036 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.09076886699999, 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.039 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:13.040 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 102.087335599, 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:13.041 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:13.042 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:13.043 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.044 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.045 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 61.536734019000001, 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.046 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.047 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.048 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:13.049 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 50.117151395, 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.050 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.051 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:13.052 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:13.053 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.054 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:13.055 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:13.068 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:13.066+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:13.158 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:13.158 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:13.158 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:13.158 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:13.167 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:13.168 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:13.166+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:13.269 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:13.269 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:13.269 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:13.270 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:13.270 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:13.270 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:13.267+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:13.277 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:13.370 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:13.369+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:13.422 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:13.422 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 168.001659913, 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.423 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:13.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:13.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:13.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.429 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.97831506700001, 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.430 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:13.432 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.433 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:13.434 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.87694808399999, 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.436 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:13.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:13.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.87676426600001, 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.443 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:13.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.449 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.876725421, 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.876673447, 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.456 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:13.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:13.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.87665771900001, 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.462 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:13.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:13.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.876639071, 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.468 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:13.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:13.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:13.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.87658112, 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.475 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:13.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:13.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:13.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.87656328899999, 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:13.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:13.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.87648226300001, 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.487 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:13.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:13.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.87638100800001, 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:13.493 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:13.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:13.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:13.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:13.499 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.87633437900001, 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:13.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:13.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:13.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.41899052599999, 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.506 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:13.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.41892133600001, 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.512 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.515 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:13.516 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.517 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.41865608000001, 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.518 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.519 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:13.520 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.521 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.522 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:13.523 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:13.524 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.41858181000001, 2024-09-15T21:34:13.525 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:13.525 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.525 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.525 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.525 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.528 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.529 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:13.530 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.531 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:13.532 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.533 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.418552909, 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.534 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.535 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.536 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.537 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:13.539 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.41853758100001, 2024-09-15T21:34:13.540 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:13.541 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.542 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:13.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.546 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.418453293, 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.547 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:13.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:13.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:13.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.552 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:13.553 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:13.553 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:13.554 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:13.555 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:13.552+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:13.655 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:13.654+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:13.675 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:13.675 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:13.675 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:13.675 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:13.684 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:13.757 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:13.755+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:13.792 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:13.793 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:13.793 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:13.793 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:13.793 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:13.801 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:13.857 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:13.855+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:13.950 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 167.507117704, 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.951 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:13.953 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:13.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 167.506204672, 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.956 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:13.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 167.40033639999999, 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:13.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:13.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 167.39992412999999, 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:13.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:13.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.970 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 167.39947596100001, 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:13.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:13.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:13.975 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 167.39863761300001, 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:13.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:13.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.980 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 167.397721147, 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.981 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:13.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:13.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:13.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.94729357700001, 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:13.989 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.991 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.94684681800001, 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:13.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:13.996 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.94641957600001, 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:13.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:13.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:14.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.001 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.945952627, 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:14.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:14.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:14.006 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.94558408399999, 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:14.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.94516701399999, 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.011 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:14.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.942515998, 2024-09-15T21:34:14.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:14.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:14.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.94181866900001, 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.021 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.022 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:14.023 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.024 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:14.025 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.65849377500001, 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.026 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:14.027 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.028 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:14.029 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.030 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.65450384499999, 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.031 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.032 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:14.033 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.034 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.035 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.65384758299999, 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.036 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.037 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.038 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.039 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.040 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.64939402499999, 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:14.041 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.042 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.043 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.044 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:14.045 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.64666127000001, 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.046 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:14.047 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.048 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.049 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.050 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:14.051 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:14.052 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:14.050+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:14.153 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:14.151+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:14.160 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:14.160 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:14.160 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:14.160 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:14.169 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:14.254 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:14.252+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:14.316 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:14.316 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:14.316 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:14.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:14.354 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:14.353+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 161.97298670500001, 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:14.432 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:14.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:14.436 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 160.96710531299999, 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:14.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:14.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 21.448585916999999, 2024-09-15T21:34:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:14.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.447 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:14.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:14.455 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:14.453+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:14.555 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:14.556 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:14.556 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:14.556 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:14.554+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:14.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:14.656 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:14.655+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:14.712 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:14.712 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:14.712 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:14.720 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:14.762 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:14.760+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:14.858 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:14.857+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:14.868 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:14.868 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 163.41109351399999, 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.869 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.870 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:14.871 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.872 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.873 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 77.748703117000005, 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.874 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:14.875 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.876 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:14.877 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.878 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 63.378669207999998, 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.879 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.880 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.881 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:14.882 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:14.883 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.884 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 57.650181715999999, 2024-09-15T21:34:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:14.885 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:14.886 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.887 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:14.888 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:14.889 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:14.890 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:14.891 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:14.892 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.893 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:14.894 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:14.894 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:14.896 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:14.960 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:14.958+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:15.004 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:15.005 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:15.005 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:15.013 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:15.060 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:15.058+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:15.121 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:15.121 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:15.121 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:15.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:15.161 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:15.160+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:15.263 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:15.261+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:15.277 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.277 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 78.158199916000001, 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:15.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:34:15.279 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:15.280 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:15.281 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:15.282 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:15.282 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:15.286 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:15.364 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:15.362+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:15.401 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:15.401 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:15.401 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:15.401 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:15.410 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:15.495 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:15.493+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:15.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:15.554 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:15.555 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:15.555 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:15.555 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:15.555 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:15.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:15.604 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:15.602+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:15.666 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:15.664+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:15.707 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:15.707 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:15.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:15.707 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:34:15.707 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:15.715 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:15.767 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:15.765+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:15.859 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:15.859 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:15.859 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:15.859 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:15.868 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:15.868 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:15.866+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:15.969 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:15.967+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:16.012 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:16.012 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:16.012 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:16.021 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:16.070 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:16.068+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.669385484, 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:16.124 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.125 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:16.126 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.127 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.128 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:16.129 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.130 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.663245854, 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:16.131 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.132 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.133 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:16.134 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.66007767799999, 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:16.135 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.136 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:16.137 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.138 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:16.139 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 164.65953192500001, 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.140 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.141 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.142 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.143 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.666893617, 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.144 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:16.145 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:16.146 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:16.147 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:16.148 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:16.149 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.150 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 163.66360078899999, 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.151 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:16.152 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.153 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.154 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.66105001899999, 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:16.155 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.156 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.157 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:16.158 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.159 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:16.160 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.161 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 162.65764933, 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.162 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:16.163 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.164 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.165 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.633414192000004, 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:16.166 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.167 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.168 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:16.169 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 64.630408027000001, 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.170 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.171 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.172 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:16.173 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.174 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.174 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:16.175 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:16.176 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:16.173+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:16.274 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:16.274 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T21:34:16.275 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:16.274+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:16.278 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:16.278 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:16.279 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:16.279 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:16.287 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:16.376 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:16.375+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:16.390 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:16.390 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:16.390 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:16.390 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:16.390 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:16.399 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:16.477 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:16.476+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:16.542 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:16.542 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:16.542 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:16.542 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:16.542 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.542 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:16.542 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 113.627625106, 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.543 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.544 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:16.545 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.546 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.601287687, 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:16.547 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.548 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.549 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.550 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 105.597854419, 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:16.551 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.552 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:16.553 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.554 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.555 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:16.556 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 65.047252838999995, 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.557 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.558 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.559 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.560 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:16.561 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 53.627670215000002, 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:16.562 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.563 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:16.564 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.565 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:16.566 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:16.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.566 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.566 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.566 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.566 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:16.567 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:16.579 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:16.577+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:16.672 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:16.672 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:16.672 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:16.672 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:16.679 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:16.678+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:16.681 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:16.781 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:16.779+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:16.783 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:16.783 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:16.783 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:16.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:16.881 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:16.879+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:16.896 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:16.896 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:16.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:16.896 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:16.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.896 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 171.475356541, 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:16.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:16.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:16.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.45201169500001, 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.904 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:16.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:16.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:16.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:16.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:16.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.911 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.912 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:16.912 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.35064471199999, 2024-09-15T21:34:16.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:16.912 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.912 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.912 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.912 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.912 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:16.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:16.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:16.918 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.35046089400001, 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.919 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:16.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:16.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.350422049, 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.925 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:16.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:16.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:16.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.350370075, 2024-09-15T21:34:16.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:16.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:16.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:16.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.35035434700001, 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.938 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:16.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:16.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:16.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.350335699, 2024-09-15T21:34:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.946 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:16.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:16.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.350277748, 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:16.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:16.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:16.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.35025991699999, 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.961 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:16.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.35017889100001, 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.967 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:16.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.35007763600001, 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:16.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:16.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.979 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.980 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.980 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:16.980 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.35003100700001, 2024-09-15T21:34:16.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:16.980 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:16.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:16.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.89268715399999, 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.89261796400001, 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:16.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:16.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:16.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:16.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.892352708, 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.001 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:17.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:17.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.89227843800001, 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:17.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:17.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:17.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.892249537, 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:17.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:17.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:17.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.89223420900001, 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.019 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:17.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:17.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:17.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:17.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.892149921, 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:17.025 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:17.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:17.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:17.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:17.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:17.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:17.031 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:17.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:17.033 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:17.030+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:17.132 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:17.131+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:17.154 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:17.154 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:17.154 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:17.154 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:17.163 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:17.233 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:17.232+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:17.272 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:17.272 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:17.272 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:17.272 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:17.272 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:17.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:17.333 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:17.332+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:17.389 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:17.389 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:17.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:17.389 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:17.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.389 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.94551467599999, 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:17.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.94460164399999, 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.394 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:17.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.83873337200001, 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:17.399 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:17.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:17.403 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.404 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.83832110200001, 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:17.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.410 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.837872933, 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.415 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.837034585, 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:17.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.83611811899999, 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:17.420 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:17.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:17.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.385690549, 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:17.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:17.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.38524379, 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:17.429 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:17.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.434 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:17.434 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:17.434 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.384816548, 2024-09-15T21:34:17.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:17.434 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.434 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:17.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.38434959899999, 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:17.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.38398105600001, 2024-09-15T21:34:17.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:17.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.38356398600001, 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:17.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.38091297, 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:17.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:17.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:17.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:17.460 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.38021564100001, 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:17.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:17.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.096890747, 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:17.465 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.466 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.468 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:17.469 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.09290081699999, 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.470 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.471 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.472 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:17.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.09224455500001, 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:17.474 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.475 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.475 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.475 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.475 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:17.475 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.477 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.478 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:17.479 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.08779099700001, 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:17.480 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:17.481 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:17.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:17.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:17.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:17.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 170.085058242, 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:17.486 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:17.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.490 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.491 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:17.493 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:17.493 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:17.490+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:17.594 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:17.592+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:17.640 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:17.640 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:17.640 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:17.640 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:17.649 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:17.694 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:17.693+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:17.756 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:17.756 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:17.756 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:17.756 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:17.757 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:17.765 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:17.795 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:17.794+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:17.896 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:17.895+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:17.912 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:17.912 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:17.912 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:17.912 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:17.912 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 165.45355195600001, 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.914 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.915 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:17.916 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:17.917 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 164.44767056399999, 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.918 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:17.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.921 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 24.929151168000001, 2024-09-15T21:34:17.922 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:17.923 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.924 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:17.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.926 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:17.927 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:17.927 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:17.928 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:17.997 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:17.996+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:18.036 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:18.036 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:18.036 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:18.037 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:18.044 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:18.098 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:18.096+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:18.151 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:18.151 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:18.151 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:18.151 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:18.151 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:18.159 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:18.199 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:18.197+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:18.300 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:18.298+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:18.307 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 166.85040215500001, 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:18.308 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.309 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:18.310 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:18.311 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.312 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.188011758000002, 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.313 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:18.314 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.315 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:18.316 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 66.817977849000002, 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:18.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:18.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:18.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 61.089490357000003, 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:18.321 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.322 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:18.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.325 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 1.087505121, 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001278429, 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 5 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:18.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219037+0000", 2024-09-15T21:34:18.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1600000000000001e-06 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219049+0000", 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2153999999999999e-05 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219070+0000", 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0441e-05 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219090+0000", 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0665e-05 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220231+0000", 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001140655 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220313+0000", 2024-09-15T21:34:18.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2354000000000005e-05 2024-09-15T21:34:18.330 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.330 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:18.330 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.330 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.330 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:18.330 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:18.331 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:18.400 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:18.399+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:18.438 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:18.439 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:18.439 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:18.439 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:18.448 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:18.501 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:18.499+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:18.556 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:18.556 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:18.556 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:18.556 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:18.556 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:18.564 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:18.603 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:18.601+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:18.671 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.672 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 81.552581017999998, 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:18.673 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.674 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:34:18.676 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:34:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:18.677 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:18.677 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:18.681 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:18.726 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:18.724+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:18.796 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:18.796 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:18.796 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:18.804 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:18.805 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:18.802+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:18.905 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:18.903+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:18.949 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:18.949 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:18.949 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:18.957 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:19.006 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:19.004+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:19.100 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:19.100 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:34:19.100 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:19.106 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:19.105+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:19.109 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:19.207 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:19.205+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:19.211 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:19.211 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:19.211 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:19.212 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:19.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:19.308 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:19.306+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:19.322 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:19.322 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:19.322 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:19.331 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:19.408 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:19.406+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:19.434 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:19.434 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.980115531, 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:19.435 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:19.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:19.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:19.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:19.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.97397590099999, 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.97080772499999, 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 167.970261972, 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:19.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.97762366399999, 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.455 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:19.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:19.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:19.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:19.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:19.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:19.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:19.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:19.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:19.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:19.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:19.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:19.463 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.464 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.464 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.464 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:19.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:19.465 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 166.97433083600001, 2024-09-15T21:34:19.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:19.465 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.465 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.466 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:19.466 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:19.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:19.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:19.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:19.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:19.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:19.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:19.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:19.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:19.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:19.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:19.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:19.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:19.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:19.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:19.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:19.475 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.475 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.476 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.476 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:19.476 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:19.476 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 165.97178006600001, 2024-09-15T21:34:19.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:19.477 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.477 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.477 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:19.477 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:19.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:19.478 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:19.478 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:19.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.478 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:19.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:19.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:19.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:19.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:19.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:19.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:19.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:19.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:19.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:19.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:19.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:19.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:19.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:19.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:19.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:19.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:19.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:19.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:19.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:19.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:19.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:19.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:19.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:19.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:19.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:19.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:19.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:19.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:19.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:19.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:19.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 165.96837937699999, 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:19.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:19.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.944144238999996, 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:19.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 67.941138073999994, 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.506 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.506 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.506 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:19.507 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:19.509 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:19.507+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:19.610 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:19.608+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:19.611 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:19.611 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:19.611 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:19.619 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:19.711 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:19.709+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:19.721 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:19.721 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:19.721 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:19.721 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:19.721 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:19.730 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:19.811 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:19.809+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:19.833 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 116.919028286, 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.892690867, 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:19.838 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:19.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.842 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 108.889257599, 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:19.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:19.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:19.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 68.338656018999998, 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:19.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.851 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 56.919073394999998, 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:19.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:19.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:19.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:19.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:19.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:19.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.856 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.856 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:19.856 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:19.856 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:19.857 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:19.941 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:19.939+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:20.001 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:20.001 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:20.002 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:20.002 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:20.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:20.014 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:20.012+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:20.035 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:20.035 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:20.053 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:20.054 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:20.071 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:20.071 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:20.088 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:20.088 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:20.114 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:20.112+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:20.121 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:20.121 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:20.153 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:20.153 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:20.153 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:20.153 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:20.153 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:20.156 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:20.156 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:20.162 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:20.178 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:20.178 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:20.194 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:20.195 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:20.247 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:20.245+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:20.264 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:20.265 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:20.273 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:20.317 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:20.316+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:20.333 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":62,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:20.349 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-09-15T21:34:20.349 INFO:tasks.mon_recovery:causing some monitor log activity 2024-09-15T21:34:20.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '1 of 30' 2024-09-15T21:34:20.382 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:20.383 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:20.392 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:20.435 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:20.433+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:20.518 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:20.517+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:20.550 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:20.559 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:20.619 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:20.618+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:20.670 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:20.671 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:20.680 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:20.725 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:20.723+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:20.787 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:20.788 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:20.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:20.822 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:20.819+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:20.922 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:20.921+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:20.947 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:20.947 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:20.956 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:21.016 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:34:21.023 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:21.021+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:21.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '2 of 30' 2024-09-15T21:34:21.069 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:21.069 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:21.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:21.124 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:21.123+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:21.224 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:21.222+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:21.232 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:21.232 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:21.241 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:21.325 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:21.324+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:21.389 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:21.389 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:21.398 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:21.426 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:21.425+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:21.504 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:21.504 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:21.513 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:21.557 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:21.555+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:21.620 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:21.621 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:21.628 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:21.627+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:21.629 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:21.728 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:21.727+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:21.777 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:21.777 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:21.785 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:21.829 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:21.828+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:21.894 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:21.894 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:21.903 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:21.947 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:21.946+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:22.017 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T21:34:22.020 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:22.020 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:22.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '3 of 30' 2024-09-15T21:34:22.030 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:22.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T21:34:22.032 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:22.030+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:22.131 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:22.129+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:22.175 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:22.175 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:22.183 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:22.206 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:22.232 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:22.231+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:22.285 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:22.286 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:22.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:22.333 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:22.331+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:22.333 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:22.334 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:22.352 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:22.352 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:22.370 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:22.370 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:22.386 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:22.386 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:22.403 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:22.403 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:22.420 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:22.421 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:22.434 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:22.433+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:22.437 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:22.437 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:22.441 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:22.441 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:22.441 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:22.450 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:22.454 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:22.454 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:22.535 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:22.534+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:22.594 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:22.594 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:22.595 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:22.595 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:22.603 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:22.644 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:22.643+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:22.737 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:22.736+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 171.291517024, 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:22.746 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.747 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.748 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:22.749 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.750 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.751 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:22.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:22.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:22.752 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.752 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.752 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:22.752 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:22.752 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.753 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 171.28537739399999, 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:22.754 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.755 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:22.755 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.755 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.755 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.755 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:22.756 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:22.757 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.758 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 171.28220921799999, 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.759 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.760 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:22.761 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.762 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:22.763 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 171.28166346500001, 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.764 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:22.765 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:22.766 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.767 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.289025157, 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:22.768 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.769 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.770 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:22.771 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.772 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 170.28573232900001, 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:22.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:22.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.28318155900001, 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:22.779 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.780 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:22.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:22.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 169.27978087, 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.786 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:22.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:22.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:22.790 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.255545732000002, 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:22.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.792 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:22.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.252539566999999, 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:22.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:22.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.799 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.799 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:22.799 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:22.799 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:22.800 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:22.841 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:22.840+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:22.939 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:22.938+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:22.944 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:22.944 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:22.944 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:22.944 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:22.953 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:23.018 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T21:34:23.031 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '4 of 30' 2024-09-15T21:34:23.032 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:34:23.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T21:34:23.078 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:23.076+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:23.096 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:23.096 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:23.097 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:23.097 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:23.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:23.149 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:23.147+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 120.295452965, 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.210 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.211 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:23.212 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.213 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:23.214 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.26911554599999, 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.215 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.216 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.217 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.218 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 112.265682278, 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.219 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:23.220 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.221 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.222 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 71.715080697999994, 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.223 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.224 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.225 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.226 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 60.295498074000001, 2024-09-15T21:34:23.227 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.228 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.229 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.230 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.231 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.232 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.232 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.232 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:23.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:23.274 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:23.272+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:23.342 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:23.340+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:23.375 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:23.376 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:23.376 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:23.376 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:23.384 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:23.443 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:23.441+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:23.486 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:23.486 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:23.487 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:23.487 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:23.495 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:23.543 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:23.542+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:23.599 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:23.599 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 178.17846228400001, 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:23.600 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.601 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:23.602 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:23.603 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.604 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:23.605 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.606 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.15511743799999, 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.607 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.608 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.609 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:23.610 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.611 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.612 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.053750455, 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.613 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.614 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:23.615 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.616 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.617 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.618 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05356663699999, 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:23.619 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.620 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:23.621 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.622 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.623 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.624 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05352779200001, 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.625 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.626 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:23.627 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.628 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.629 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.630 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05347581800001, 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.631 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.632 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.633 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:23.634 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:23.635 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.636 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05346008999999, 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.637 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:23.638 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.639 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.640 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:23.641 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:23.642 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05344144200001, 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.643 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.644 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:23.645 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.646 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.647 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.648 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05338349100001, 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.649 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.650 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.651 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:23.652 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:23.653 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.654 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05336566, 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.655 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:23.656 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.657 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.658 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.659 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.660 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05328463399999, 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.661 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:23.662 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.663 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.664 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:23.665 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.666 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05318337899999, 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.667 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:23.668 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:23.669 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.670 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:23.671 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.672 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 177.05313674999999, 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:23.673 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.674 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.675 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:23.676 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.677 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.678 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 176.595792897, 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.679 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.680 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:23.681 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:23.682 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.683 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:23.684 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 176.59572370699999, 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.685 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.686 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:23.687 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:23.688 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.689 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.690 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 176.59545845100001, 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.691 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.692 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:23.693 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:23.694 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.695 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.696 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 176.59538418099999, 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:23.697 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.698 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.699 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:23.700 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:23.701 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.702 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 176.59535528000001, 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.703 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.704 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:23.705 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:23.706 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.707 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.708 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 176.59533995199999, 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.709 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.710 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:23.711 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.712 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.713 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:23.714 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 176.59525566400001, 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:23.715 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:23.716 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:23.717 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.718 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:23.719 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:23.720 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:23.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.721 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.721 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:23.721 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:23.721 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:23.722 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:23.723 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:23.720+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:23.823 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:23.822+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:23.835 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:23.835 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:23.836 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:23.845 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:23.924 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:23.922+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:23.952 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:23.952 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:23.952 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:23.961 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:24.023 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T21:34:24.025 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:24.023+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:24.025 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:34:24.029 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '5 of 30' 2024-09-15T21:34:24.040 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T21:34:24.077 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:24.077 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.63325210400001, 2024-09-15T21:34:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.078 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.079 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.080 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:24.081 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.082 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.083 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.084 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.085 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:24.086 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.63233907200001, 2024-09-15T21:34:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.087 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:24.088 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.089 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:24.090 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.091 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:24.092 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.093 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.094 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.5264708, 2024-09-15T21:34:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.095 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:24.096 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.097 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:24.098 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.099 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:24.100 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.101 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:24.102 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.52605853, 2024-09-15T21:34:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.103 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.104 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.105 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:24.106 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.107 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.108 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:24.109 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.52561036099999, 2024-09-15T21:34:24.110 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.111 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.112 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.113 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.114 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:24.115 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.116 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:24.117 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:24.118 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.52477201299999, 2024-09-15T21:34:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.119 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.120 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:24.121 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.122 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:24.123 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:24.124 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.125 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.52385554700001, 2024-09-15T21:34:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.126 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.127 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.128 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.129 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:24.130 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.131 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:24.132 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.07342797699999, 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.133 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:24.134 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.135 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.136 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.137 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.072981218, 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.138 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.139 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:24.140 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:24.141 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.07255397599999, 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.142 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:24.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.146 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.07208702700001, 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:24.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.149 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.150 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.071718484, 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.151 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.152 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:24.153 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:24.154 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.155 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.071301414, 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:24.156 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.157 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.158 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:24.159 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.06865039799999, 2024-09-15T21:34:24.160 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.161 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:24.162 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.163 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:24.164 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 177.067953069, 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.165 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:24.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.170 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 176.78462817499999, 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.171 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.172 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:24.173 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.174 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:24.175 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 176.78063824500001, 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.176 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.177 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:24.178 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.179 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 176.779981983, 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.180 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.181 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.182 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:24.183 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.184 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 176.775528425, 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.185 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:24.186 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.187 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.188 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.189 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:24.190 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 176.77279566999999, 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.191 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.192 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.193 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:24.194 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:24.195 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.196 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.196 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:24.196 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:24.231 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:24.229+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:24.299 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:24.297+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:24.339 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:24.340 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:24.340 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:24.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:24.400 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:24.398+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:24.456 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:24.456 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:24.457 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:24.457 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:24.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:24.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:24.501 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:24.499+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:24.601 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:24.600+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:24.615 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:24.615 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:24.615 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:24.615 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 172.156292562, 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.616 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.617 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.618 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:24.619 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.620 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 171.15041117000001, 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.621 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.622 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:24.623 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.624 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.625 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 31.631891774, 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.626 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:24.627 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:24.628 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.629 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.630 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:24.630 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:24.630 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:24.631 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:24.637 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:24.637 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:24.655 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:24.655 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:24.672 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:24.672 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:24.688 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:24.689 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:24.702 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:24.700+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:24.706 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:24.706 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:24.723 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:24.724 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:24.740 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:24.741 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:24.746 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:24.746 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:24.755 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:24.758 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:24.758 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:24.803 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:24.801+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:24.865 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:24.865 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:24.874 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:24.918 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:24.916+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:24.981 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:24.981 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:24.990 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:25.026 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T21:34:25.027 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:25.025+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:25.030 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '6 of 30' 2024-09-15T21:34:25.041 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T21:34:25.126 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:25.124+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:25.137 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:25.137 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:25.146 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:25.207 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:25.205+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:25.296 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:25.296 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:25.305 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:25.308 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:25.306+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:25.409 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:25.408+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:25.412 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:25.413 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:25.422 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:25.510 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:25.508+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:25.538 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:25.538 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:25.547 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:25.633 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:25.630+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:25.692 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:25.693 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:25.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:25.712 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:25.710+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:25.813 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:25.811+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:25.844 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:25.845 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:25.853 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:25.913 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:25.912+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:25.955 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:25.956 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:25.965 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:26.015 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:26.013+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:26.033 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '7 of 30' 2024-09-15T21:34:26.038 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T21:34:26.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:34:26.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T21:34:26.072 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:26.072 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:26.081 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:26.140 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:26.138+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:26.184 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:26.185 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:26.194 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:26.238 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:26.236+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:26.297 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:26.297 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:26.305 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:26.346 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:26.344+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:26.419 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:26.416+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:26.448 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:26.448 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:26.457 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:26.518 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:26.517+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:26.600 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:26.600 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:26.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:26.653 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:26.651+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:26.713 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:26.713 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:26.720 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:26.717+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:26.722 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:26.770 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:26.820 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:26.818+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:26.865 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:26.865 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:26.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:26.898 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:26.898 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:26.915 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:26.915 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:26.921 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:26.920+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:26.931 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:26.931 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:26.947 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:26.947 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:26.964 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:26.965 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:26.978 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:26.978 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:26.981 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:26.981 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:26.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:26.997 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:26.998 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:27.014 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:27.015 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:27.023 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:27.021+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:27.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '8 of 30' 2024-09-15T21:34:27.046 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:34:27.046 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T21:34:27.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T21:34:27.106 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:27.106 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:27.106 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:27.106 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:27.116 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:27.124 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:27.122+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:27.224 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:27.223+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:27.267 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:27.267 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:27.268 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:27.268 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:27.268 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:27.277 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:27.361 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:27.359+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:27.426 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:27.424+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:27.427 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:27.427 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:27.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.98352202800001, 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.428 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:27.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.98260899600001, 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.434 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.435 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:27.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.876740724, 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:27.437 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.438 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:27.439 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.440 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.441 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.876328454, 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.442 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:27.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.446 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.87588028499999, 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:27.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.87504193699999, 2024-09-15T21:34:27.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:27.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.87412547100001, 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.457 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:27.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.461 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.423697901, 2024-09-15T21:34:27.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.464 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.465 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:27.466 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.467 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.423251142, 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.469 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.470 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.472 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.4228239, 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:27.474 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.475 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:27.476 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:27.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.42235695100001, 2024-09-15T21:34:27.478 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.480 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:27.481 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:27.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.483 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.421988408, 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.484 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.487 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:27.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.489 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.42157133800001, 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.492 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.493 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:27.494 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.41892032199999, 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.495 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:27.496 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:27.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.498 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.499 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.500 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.418222993, 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.501 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:27.502 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:27.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.504 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.505 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.134898099, 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.506 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:27.507 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:27.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.509 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.510 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.511 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.13090816900001, 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:27.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.130251907, 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:27.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:27.522 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.12579834900001, 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.123065594, 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.528 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:27.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.531 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:27.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:27.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.533 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.533 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.533 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.533 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:27.535 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:27.535 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:27.532+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:27.636 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:27.633+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:27.683 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:27.683 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:27.683 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:27.683 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:27.691 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:27.736 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:27.734+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:27.799 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:27.799 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:27.799 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:27.799 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:27.799 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:27.807 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:27.849 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:27.847+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:27.938 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:27.936+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:27.955 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:27.955 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:27.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 175.49687576599999, 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:27.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:27.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 174.490994374, 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:27.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 34.972474978000001, 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.966 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:27.967 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:27.969 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:27.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:27.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.970 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.970 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:27.970 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:27.970 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:27.971 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:28.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '9 of 30' 2024-09-15T21:34:28.039 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:28.037+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:28.048 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T21:34:28.063 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T21:34:28.123 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:28.123 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:28.123 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:28.123 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:28.134 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:28.176 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:28.174+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:28.241 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:28.239+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:28.287 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:28.287 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:28.287 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:28.287 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:28.287 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:28.297 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:28.343 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:28.340+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:28.405 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:28.405 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:28.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:28.405 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:28.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.405 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 176.94787223700001, 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:28.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:28.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:28.410 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 91.285481840000003, 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:28.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 76.915447931000003, 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:28.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 71.186960439000003, 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:28.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:28.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:28.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 11.184975203, 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001278429, 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:28.423 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 5 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219037+0000", 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1600000000000001e-06 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219049+0000", 2024-09-15T21:34:28.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2153999999999999e-05 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219070+0000", 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0441e-05 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219090+0000", 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0665e-05 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220231+0000", 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001140655 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220313+0000", 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2354000000000005e-05 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.427 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.428 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:28.428 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:28.429 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:28.443 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:28.441+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:28.536 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:28.536 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:28.536 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:28.536 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:28.543 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:28.542+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:28.545 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:28.645 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:28.643+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:28.692 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:28.692 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:28.692 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:28.692 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:28.692 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:28.701 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:28.745 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:28.743+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:28.808 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:28.808 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 91.689378587999997, 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:28.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:28.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:34:28.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:28.813 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:28.814 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:28.817 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:28.859 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:28.857+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:28.933 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:28.933 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:28.933 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:28.933 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:28.942 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:28.947 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:28.946+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:29.027 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:29.034 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '10 of 30' 2024-09-15T21:34:29.046 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:29.046 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:29.046 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:29.046 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:29.046 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:29.048 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:29.046+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:29.049 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T21:34:29.055 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:29.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:34:29.065 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T21:34:29.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:29.151 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:29.149+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:29.162 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:29.163 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:29.197 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:29.197 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:29.201 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:29.201 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:29.201 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:29.201 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:34:29.201 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:29.210 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:29.230 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:29.230 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:29.252 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:29.253 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:29.254 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:29.252+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:29.269 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:29.270 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:29.286 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:29.286 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:29.303 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:29.303 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:29.315 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:29.315 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:29.320 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:29.321 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:29.324 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:29.374 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:29.372+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:29.439 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:34:29.440 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:34:29.452 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-15T21:34:29.048613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221720,"kb_used_data":5480,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753000,"statfs":{"total":773094113280,"available":772867072000,"internally_reserved":0,"allocated":5611520,"data_stored":3347836,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022299"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":111,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:34:27.047162+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:34:27.047162+0000","last_peered":"2024-09-15T21:34:27.047162+0000","last_clean":"2024-09-15T21:34:27.047162+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:34:27.047162+0000","last_undegraded":"2024-09-15T21:34:27.047162+0000","last_fullsized":"2024-09-15T21:34:27.047162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.029530+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:34:25.029530+0000","last_peered":"2024-09-15T21:34:25.029530+0000","last_clean":"2024-09-15T21:34:25.029530+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:34:25.029530+0000","last_undegraded":"2024-09-15T21:34:25.029530+0000","last_fullsized":"2024-09-15T21:34:25.029530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:34:23.858531+0000","last_change":"2024-09-15T21:33:29.803626+0000","last_active":"2024-09-15T21:34:23.858531+0000","last_peered":"2024-09-15T21:34:23.858531+0000","last_clean":"2024-09-15T21:34:23.858531+0000","last_became_active":"2024-09-15T21:33:29.803416+0000","last_became_peered":"2024-09-15T21:33:29.803416+0000","last_unstale":"2024-09-15T21:34:23.858531+0000","last_undegraded":"2024-09-15T21:34:23.858531+0000","last_fullsized":"2024-09-15T21:34:23.858531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:13.277699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.029664+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:34:25.029664+0000","last_peered":"2024-09-15T21:34:25.029664+0000","last_clean":"2024-09-15T21:34:25.029664+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:34:25.029664+0000","last_undegraded":"2024-09-15T21:34:25.029664+0000","last_fullsized":"2024-09-15T21:34:25.029664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:34:28.048861+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:34:28.048861+0000","last_peered":"2024-09-15T21:34:28.048861+0000","last_clean":"2024-09-15T21:34:28.048861+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:34:28.048861+0000","last_undegraded":"2024-09-15T21:34:28.048861+0000","last_fullsized":"2024-09-15T21:34:28.048861+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:34:26.584426+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:34:26.584426+0000","last_peered":"2024-09-15T21:34:26.584426+0000","last_clean":"2024-09-15T21:34:26.584426+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:34:26.584426+0000","last_undegraded":"2024-09-15T21:34:26.584426+0000","last_fullsized":"2024-09-15T21:34:26.584426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:34:23.858517+0000","last_change":"2024-09-15T21:33:30.398919+0000","last_active":"2024-09-15T21:34:23.858517+0000","last_peered":"2024-09-15T21:34:23.858517+0000","last_clean":"2024-09-15T21:34:23.858517+0000","last_became_active":"2024-09-15T21:33:30.398643+0000","last_became_peered":"2024-09-15T21:33:30.398643+0000","last_unstale":"2024-09-15T21:34:23.858517+0000","last_undegraded":"2024-09-15T21:34:23.858517+0000","last_fullsized":"2024-09-15T21:34:23.858517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:19.700973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:34:28.048861+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:34:28.048861+0000","last_peered":"2024-09-15T21:34:28.048861+0000","last_clean":"2024-09-15T21:34:28.048861+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:34:28.048861+0000","last_undegraded":"2024-09-15T21:34:28.048861+0000","last_fullsized":"2024-09-15T21:34:28.048861+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":45,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:34:23.858525+0000","last_change":"2024-09-15T21:33:29.811291+0000","last_active":"2024-09-15T21:34:23.858525+0000","last_peered":"2024-09-15T21:34:23.858525+0000","last_clean":"2024-09-15T21:34:23.858525+0000","last_became_active":"2024-09-15T21:33:29.803834+0000","last_became_peered":"2024-09-15T21:33:29.803834+0000","last_unstale":"2024-09-15T21:34:23.858525+0000","last_undegraded":"2024-09-15T21:34:23.858525+0000","last_fullsized":"2024-09-15T21:34:23.858525+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:54.379060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476776,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":700416,"data_stored":802227,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":284199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476777,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":284218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476777,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":724992,"data_stored":807677,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":295118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":708608,"data_stored":295099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":289649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":289649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:29.453 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:29.453 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:29.450+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:29.467 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:29.467 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:29.476 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:29.553 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:29.551+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:29.581 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:29.581 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:29.590 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:29.654 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:29.652+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:29.693 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:29.693 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:29.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:29.785 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:29.783+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:29.785 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:34:29.786 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:34:29.799 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-15T21:34:29.048613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":221720,"kb_used_data":5480,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753000,"statfs":{"total":773094113280,"available":772867072000,"internally_reserved":0,"allocated":5611520,"data_stored":3347836,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022299"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":111,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-09-15T21:34:27.047162+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:34:27.047162+0000","last_peered":"2024-09-15T21:34:27.047162+0000","last_clean":"2024-09-15T21:34:27.047162+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:34:27.047162+0000","last_undegraded":"2024-09-15T21:34:27.047162+0000","last_fullsized":"2024-09-15T21:34:27.047162+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.029530+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:34:25.029530+0000","last_peered":"2024-09-15T21:34:25.029530+0000","last_clean":"2024-09-15T21:34:25.029530+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:34:25.029530+0000","last_undegraded":"2024-09-15T21:34:25.029530+0000","last_fullsized":"2024-09-15T21:34:25.029530+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:34:23.858531+0000","last_change":"2024-09-15T21:33:29.803626+0000","last_active":"2024-09-15T21:34:23.858531+0000","last_peered":"2024-09-15T21:34:23.858531+0000","last_clean":"2024-09-15T21:34:23.858531+0000","last_became_active":"2024-09-15T21:33:29.803416+0000","last_became_peered":"2024-09-15T21:33:29.803416+0000","last_unstale":"2024-09-15T21:34:23.858531+0000","last_undegraded":"2024-09-15T21:34:23.858531+0000","last_fullsized":"2024-09-15T21:34:23.858531+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:37:13.277699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.029664+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:34:25.029664+0000","last_peered":"2024-09-15T21:34:25.029664+0000","last_clean":"2024-09-15T21:34:25.029664+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:34:25.029664+0000","last_undegraded":"2024-09-15T21:34:25.029664+0000","last_fullsized":"2024-09-15T21:34:25.029664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":69,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:34:28.048861+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:34:28.048861+0000","last_peered":"2024-09-15T21:34:28.048861+0000","last_clean":"2024-09-15T21:34:28.048861+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:34:28.048861+0000","last_undegraded":"2024-09-15T21:34:28.048861+0000","last_fullsized":"2024-09-15T21:34:28.048861+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":97,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:34:26.584426+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:34:26.584426+0000","last_peered":"2024-09-15T21:34:26.584426+0000","last_clean":"2024-09-15T21:34:26.584426+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:34:26.584426+0000","last_undegraded":"2024-09-15T21:34:26.584426+0000","last_fullsized":"2024-09-15T21:34:26.584426+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":39,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:34:23.858517+0000","last_change":"2024-09-15T21:33:30.398919+0000","last_active":"2024-09-15T21:34:23.858517+0000","last_peered":"2024-09-15T21:34:23.858517+0000","last_clean":"2024-09-15T21:34:23.858517+0000","last_became_active":"2024-09-15T21:33:30.398643+0000","last_became_peered":"2024-09-15T21:33:30.398643+0000","last_unstale":"2024-09-15T21:34:23.858517+0000","last_undegraded":"2024-09-15T21:34:23.858517+0000","last_fullsized":"2024-09-15T21:34:23.858517+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:19.700973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:34:28.048861+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:34:28.048861+0000","last_peered":"2024-09-15T21:34:28.048861+0000","last_clean":"2024-09-15T21:34:28.048861+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:34:28.048861+0000","last_undegraded":"2024-09-15T21:34:28.048861+0000","last_fullsized":"2024-09-15T21:34:28.048861+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":45,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-15T21:34:23.858525+0000","last_change":"2024-09-15T21:33:29.811291+0000","last_active":"2024-09-15T21:34:23.858525+0000","last_peered":"2024-09-15T21:34:23.858525+0000","last_clean":"2024-09-15T21:34:23.858525+0000","last_became_active":"2024-09-15T21:33:29.803834+0000","last_became_peered":"2024-09-15T21:33:29.803834+0000","last_unstale":"2024-09-15T21:34:23.858525+0000","last_undegraded":"2024-09-15T21:34:23.858525+0000","last_fullsized":"2024-09-15T21:34:23.858525+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:32:54.379060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476776,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":700416,"data_stored":802227,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":684032,"data_stored":284199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476777,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":688128,"data_stored":284218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476777,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":724992,"data_stored":807677,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542185,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":712704,"data_stored":295118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27724,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344116,"statfs":{"total":96636764160,"available":96608374784,"internally_reserved":0,"allocated":708608,"data_stored":295099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":696320,"data_stored":289649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":289649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:29.800 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T21:34:29.800 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:29.801 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-15T21:34:29.801 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 0.000000 2024-09-15T21:34:29.801 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 0 2024-09-15T21:34:29.844 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:29.844 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:29.852 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:29.894 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:29.892+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:29.957 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:29.955+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:29.996 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:29.996 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:30.006 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:30.035 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '11 of 30' 2024-09-15T21:34:30.055 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:34:30.058 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:30.056+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:30.149 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:30.150 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:30.158 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:30.159 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:30.158+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:30.260 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:30.258+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:30.260 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:30.261 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:30.269 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:30.361 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:30.359+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:30.413 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:30.413 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:30.422 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:30.466 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:30.464+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:30.530 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:30.530 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:30.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:30.581 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:30.579+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:30.663 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:30.661+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:30.687 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:30.696 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:30.781 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:30.779+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:30.844 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:30.844 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:30.853 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:30.864 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:30.862+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:30.965 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:30.963+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:31.000 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:31.000 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:31.009 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:31.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '12 of 30' 2024-09-15T21:34:31.056 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.7 primary-affinity to 0 (802) 2024-09-15T21:34:31.065 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:31.064+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:31.121 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:31.121 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:31.130 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:31.166 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:31.164+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:31.243 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:31.244 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:31.253 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:31.268 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:31.266+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:31.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:31.362 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:31.362 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:31.368 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:31.366+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:31.371 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:31.461 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:31.462 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:31.469 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:31.467+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:31.478 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:31.478 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:31.495 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:31.495 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:31.511 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:31.512 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:31.523 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:31.523 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:31.523 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:31.523 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:31.523 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:31.529 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:31.529 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:31.532 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:31.546 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:31.547 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:31.564 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:31.564 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:31.569 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:31.567+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:31.581 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:31.582 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:31.655 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:31.655 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:31.655 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:31.655 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 180.19784736899999, 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:31.656 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.657 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:31.658 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:31.659 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:31.660 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 94.535456972000006, 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.661 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.662 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:31.663 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:31.664 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 80.165423063000006, 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:31.665 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.666 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:31.667 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 74.436935571000006, 2024-09-15T21:34:31.668 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:31.669 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.670 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:31.671 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.672 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 14.434950335, 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001278429, 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:31.673 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 5 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:31.674 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219037+0000", 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1600000000000001e-06 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219049+0000", 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2153999999999999e-05 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.675 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219070+0000", 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0441e-05 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219090+0000", 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0665e-05 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220231+0000", 2024-09-15T21:34:31.676 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001140655 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220313+0000", 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2354000000000005e-05 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:31.677 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:31.679 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:31.679 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:31.677+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:31.779 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:31.778+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:31.826 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:31.827 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:31.827 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:31.827 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:31.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:31.881 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:31.879+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:31.982 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:31.980+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:31.983 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:31.983 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:31.983 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:31.983 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:31.983 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:31.992 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:32.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '13 of 30' 2024-09-15T21:34:32.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:34:32.082 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:32.080+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:32.143 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:32.143 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:32.143 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:32.143 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:32.143 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.143 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 95.023781416000006, 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:32.144 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:34:32.145 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:34:32.146 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:34:32.147 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:32.148 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:34:32.149 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:34:32.149 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:32.149 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:32.149 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:32.149 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:32.149 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:32.149 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:32.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:32.194 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:32.192+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:32.267 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:32.268 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:32.268 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:32.277 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:32.321 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:32.319+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:32.381 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:32.381 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:32.381 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:32.384 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:32.383+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:32.389 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:32.486 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:32.484+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:32.533 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:32.533 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:32.533 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:32.533 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:34:32.534 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:32.543 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:32.586 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:32.585+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:32.686 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:32.686 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:32.686 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:32.686 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:32.688 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:32.686+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:32.695 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:32.719 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:34:32.788 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:32.786+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:32.798 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:32.798 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:32.798 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:32.807 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:32.889 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:32.888+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.49713088499999, 2024-09-15T21:34:32.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:32.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:32.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:32.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.49099125500001, 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:32.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:32.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.48782307900001, 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.964 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:32.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:32.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:32.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:32.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:32.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:32.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.968 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 181.487277326, 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:32.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:32.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:32.973 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 180.49463901799999, 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:32.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:32.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:32.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.980 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 180.49134619, 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:32.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:32.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 179.48879542, 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.985 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:32.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:32.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:32.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 179.48539473100001, 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.992 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.461159593000005, 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:32.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:32.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.458153428000003, 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:33.001 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:33.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-15T21:34:33.005 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 10.035178975000001, 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023751800000000001, 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914744+0000", 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5999999999999999e-06 2024-09-15T21:34:33.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914750+0000", 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914759+0000", 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2e-06 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914768+0000", 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4689999999999997e-06 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914858+0000", 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0499000000000002e-05 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914980+0000", 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012141099999999999 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.009 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.010 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.010 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:33.011 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:33.012 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:33.009+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:33.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '14 of 30' 2024-09-15T21:34:33.115 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:33.116 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:33.116 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:33.124 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:33.129 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:33.127+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:33.212 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:33.210+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:33.227 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:33.227 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:33.227 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:33.236 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:33.314 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:33.311+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:33.339 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 130.42541299000001, 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.340 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:33.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.344 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.399075571, 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:33.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:33.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 122.395642303, 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 81.845040722999997, 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 70.425458098999997, 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.358 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:33.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.362 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.362 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:33.363 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:33.447 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:33.445+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:33.507 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:33.507 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:33.507 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:33.507 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:33.515 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:33.513+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:33.516 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:33.595 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:33.639 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:33.638+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:33.659 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:33.659 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:33.659 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:33.659 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:33.659 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:33.668 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:33.717 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:33.715+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:33.720 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:33.721 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:33.738 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:33.738 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:33.753 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:33.754 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:33.769 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:33.770 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 188.35144424399999, 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.773 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.774 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:33.775 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:33.776 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.777 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:33.778 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.779 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.32809939800001, 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.780 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.781 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.782 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.783 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:33.784 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.785 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.22673241499999, 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.786 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.787 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.788 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.789 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.790 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.791 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.226548597, 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.792 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:33.793 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.794 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.795 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:33.796 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.797 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.226509752, 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:33.798 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:33.799 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:33.800 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.801 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.802 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:33.803 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.226457778, 2024-09-15T21:34:33.804 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.805 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:33.806 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.807 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.808 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:33.809 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.22644205, 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:33.810 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.811 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.812 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:33.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.22642340199999, 2024-09-15T21:34:33.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:33.818 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.818 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.818 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.818 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.818 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:33.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:33.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:33.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:33.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.22636545099999, 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:33.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:33.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:33.831 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.22634762000001, 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:33.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:33.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.837 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.22626659400001, 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.838 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:33.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:33.844 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.845 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.845 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.845 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.845 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:33.846 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.226165339, 2024-09-15T21:34:33.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:33.846 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.846 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.846 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.847 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:33.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.847 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:33.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:33.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:33.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:33.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:33.851 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:33.851 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.851 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.851 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.851 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:33.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:33.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:33.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:33.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:33.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:33.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.854 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.854 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.854 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:33.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:33.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:33.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:33.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:33.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:33.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:33.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:33.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:33.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:33.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:33.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:33.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.859 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.860 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:33.860 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 187.22611871000001, 2024-09-15T21:34:33.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:33.861 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.861 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.861 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.862 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.862 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:33.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.862 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:33.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:33.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:33.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:33.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:33.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:33.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.866 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:33.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:33.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:33.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:33.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:33.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:33.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:33.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:33.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:33.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:33.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:33.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:33.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:33.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:33.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:33.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:33.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.873 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.873 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:33.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:33.874 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.874 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.874 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.875 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.875 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:33.875 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.76877485700001, 2024-09-15T21:34:33.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:33.875 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.876 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.876 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.876 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.877 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:33.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.877 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:33.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:33.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:33.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:33.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:33.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:33.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.884 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.76870566700001, 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.885 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:33.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.768440411, 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:33.892 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:33.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:33.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.899 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.768366141, 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.900 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:33.906 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.76833723999999, 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.907 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:33.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:33.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:33.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.768321912, 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:33.914 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:33.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:33.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:33.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:33.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.921 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 186.76823762399999, 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:33.922 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:33.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:33.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:33.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:33.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:33.929 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:33.930 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:33.931 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:33.931 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:33.932 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:33.932 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:33.928+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:33.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:33.933 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:33.933 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:33.934 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:33.934 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:34.031 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:34.029+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:34.042 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '15 of 30' 2024-09-15T21:34:34.084 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:34.084 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:34.093 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:34.136 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:34.134+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:34.233 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:34.232+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:34.244 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:34.244 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:34.253 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:34.334 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:34.332+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:34.400 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:34.400 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:34.409 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:34.434 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:34.432+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:34.517 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:34.526 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:34.535 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:34.533+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:34.632 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:34.633 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:34.636 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:34.634+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:34.641 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:34.734 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:34:34.737 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:34.735+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:34.748 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:34.749 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:34.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:34.837 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:34.836+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:34.923 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:34.923 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:34.932 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:34.976 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:34.974+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:35.039 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:35.037+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:35.042 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:35.042 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:35.055 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '16 of 30' 2024-09-15T21:34:35.056 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:35.140 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:35.138+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:35.209 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:35.209 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:35.216 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:35.257 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:35.255+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:35.341 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:35.339+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:35.364 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:35.364 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:35.373 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:35.442 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:35.440+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:35.481 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:35.481 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:35.490 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:35.574 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:35.572+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:35.637 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:35.637 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:35.644 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:35.642+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:35.645 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:35.745 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:35.744+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:35.761 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:35.761 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:35.770 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:35.846 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:35.844+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:35.875 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:35.875 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:35.883 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:35.936 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:35.988 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:35.986+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:36.041 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:34:36.042 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:36.042 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:36.047 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:36.048 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:36.046+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:36.052 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '17 of 30' 2024-09-15T21:34:36.070 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:36.071 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:36.074 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:36.074 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:36.088 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:36.089 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:36.105 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:36.105 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:36.134 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:36.134 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:36.147 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:36.147 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:36.148 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:36.147+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:36.171 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:36.172 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:36.185 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:36.186 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:36.218 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:36.219 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:36.219 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:36.219 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:36.229 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:36.230 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:36.231 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:36.280 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:36.273+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:36.332 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:36.332 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:36.332 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:36.341 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:36.387 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:36.385+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:36.447 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:36.447 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 184.992413736, 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:36.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:36.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:36.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 184.986274106, 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:36.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:36.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.459 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 184.98310592999999, 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:36.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 184.98256017700001, 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.465 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:36.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.989921869, 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:36.470 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:36.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.473 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:36.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:36.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:36.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:36.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:36.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:36.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:36.477 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 183.98662904099999, 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:36.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 182.98407827099999, 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:36.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:36.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 182.980677582, 2024-09-15T21:34:36.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:36.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.492 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.956442444000004, 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.494 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:36.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.953436279000002, 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:36.497 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:36.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-15T21:34:36.501 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 13.530461826, 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023751800000000001, 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914744+0000", 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5999999999999999e-06 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914750+0000", 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914759+0000", 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2e-06 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914768+0000", 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4689999999999997e-06 2024-09-15T21:34:36.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914858+0000", 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0499000000000002e-05 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914980+0000", 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012141099999999999 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.505 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.506 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.506 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:36.507 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:36.508 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:36.505+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:36.607 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:36.606+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:36.652 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:36.653 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:36.653 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:36.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:36.708 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:36.706+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:36.764 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:36.765 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:36.765 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:36.765 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:36.765 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:36.773 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:36.817 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:36.815+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:36.876 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:36.876 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:36.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:36.876 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:36.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.877 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:36.877 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:36.877 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 133.96132873900001, 2024-09-15T21:34:36.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:36.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.879 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.879 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:36.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:36.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:36.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:36.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:36.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:36.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.886 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:36.886 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:36.886 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 125.93499131999999, 2024-09-15T21:34:36.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:36.886 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.886 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.887 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:36.887 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:36.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:36.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:36.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:36.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:36.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:36.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:36.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:36.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:36.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:36.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:36.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:36.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.892 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.892 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.892 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:36.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:36.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:36.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:36.893 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.894 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.894 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.894 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:36.894 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:36.894 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 125.931558052, 2024-09-15T21:34:36.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:36.895 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.895 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.895 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:36.895 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:36.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 85.380956471999994, 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:36.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:36.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 73.961373847999994, 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:36.904 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:36.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:36.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:36.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:36.909 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:36.910 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:36.911 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:36.912 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:36.909+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:37.012 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:37.010+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:37.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '18 of 30' 2024-09-15T21:34:37.057 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:37.057 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:37.057 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:37.066 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:37.151 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:37.149+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:37.210 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:37.210 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:37.211 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:37.211 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:37.211 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:37.214 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:37.213+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:37.220 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:37.316 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:37.314+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 191.945038119, 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.366 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:37.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:37.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:37.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.92169327299999, 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.373 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.374 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.374 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:37.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.374 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:37.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.82032629, 2024-09-15T21:34:37.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:37.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:37.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:37.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:37.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:37.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.82014247199999, 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.387 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:37.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:37.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:37.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.82010362700001, 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.393 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:37.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.397 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:37.399 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.82005165300001, 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:37.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:37.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.405 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.82003592500001, 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.406 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.407 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:37.408 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.409 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:37.410 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:37.410 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.410 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.410 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.411 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:37.412 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.82001727700001, 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.413 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:37.414 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.415 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.416 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:37.417 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:37.417 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.417 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.417 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.417 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:37.418 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:37.418 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.418 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.418 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.418 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:37.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:37.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.419 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.419 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:37.419 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:37.419 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.419 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:37.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:37.420 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.420 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.420 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.420 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:37.420 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:37.421 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.421 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.421 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.421 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:37.421 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:37.421 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.422 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.422 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.422 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.422 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.422 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.422 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:37.423 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.819959326, 2024-09-15T21:34:37.423 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:37.423 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.423 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.423 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.423 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.424 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:37.425 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.426 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:37.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:37.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.427 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.427 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:37.427 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:37.427 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.427 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.428 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.428 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:37.428 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:37.428 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.428 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.429 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.429 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:37.429 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:37.429 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.429 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.430 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:37.430 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:37.430 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.430 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.430 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:37.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:37.431 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.431 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.431 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.431 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:37.431 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:37.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.432 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.432 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.432 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.432 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.433 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.433 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:37.433 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.81994149499999, 2024-09-15T21:34:37.433 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:37.433 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.434 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.434 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.434 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.434 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:37.434 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.435 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:37.436 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:37.436 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.436 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.436 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.436 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:37.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:37.437 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.437 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.437 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.437 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:37.437 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:37.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.438 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:37.438 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:37.438 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.438 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.438 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:37.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:37.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.439 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.439 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.439 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:37.439 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:37.439 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.440 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.440 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:37.440 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:37.440 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.440 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:37.441 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.442 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.81986046899999, 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.443 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.444 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.445 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.446 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:37.447 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.448 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:37.449 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.81975921399999, 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.450 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.451 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.451 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:37.451 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.451 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:37.452 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.452 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.452 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.452 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.452 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.453 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:37.454 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.455 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:37.456 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.457 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.457 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.457 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:37.457 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:37.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.457 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.457 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.457 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.458 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.458 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.458 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:37.458 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.81971258499999, 2024-09-15T21:34:37.458 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:37.458 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.459 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.459 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.459 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.459 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:37.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.460 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.460 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.460 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:37.460 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.460 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.460 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:37.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.461 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.461 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.461 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.461 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:37.461 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:37.462 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:37.463 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:37.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.464 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:37.464 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:37.464 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.464 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.464 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:37.465 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.466 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.36236873199999, 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.467 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.468 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:37.469 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.470 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:37.471 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.472 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.36229954199999, 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.474 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.475 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:37.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:37.476 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.476 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.476 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.476 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:37.476 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:37.477 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:37.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.478 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:37.478 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:37.478 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.478 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.478 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:37.479 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.480 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.36203428600001, 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.481 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.482 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:37.483 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.484 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:37.485 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.486 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.36196001600001, 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.487 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:37.488 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:37.490 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:37.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.493 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.361931115, 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:37.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:37.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:37.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:37.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.36191578699999, 2024-09-15T21:34:37.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:37.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:37.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:37.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:37.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:37.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:37.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.507 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:37.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:37.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.507 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.507 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.508 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:37.508 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:37.508 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 190.361831499, 2024-09-15T21:34:37.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:37.508 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:37.509 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:37.509 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:37.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:37.509 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:37.509 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:37.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.509 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:37.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:37.511 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.512 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:37.513 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:37.514 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:37.515 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:37.515 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:37.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.515 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.515 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:37.515 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:37.515 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:37.517 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:37.514+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:37.517 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:37.617 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:37.615+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:37.625 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:37.626 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:37.626 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:37.626 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:37.634 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:37.718 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:37.716+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:37.743 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:37.743 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:37.752 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:37.819 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:37.817+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:37.901 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:37.901 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 191.45748887299999, 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.902 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.903 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.904 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.905 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:37.906 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 191.45657584099999, 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.907 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.908 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.909 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.910 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.911 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.912 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.913 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 191.35070756900001, 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.914 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.915 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:37.916 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.917 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.918 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 191.35029529900001, 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.919 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:37.920 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.921 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.922 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.923 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 191.34984713, 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:37.924 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.925 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:37.926 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.927 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.928 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 191.349008782, 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.929 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.930 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:37.932 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:37.933 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 191.34809231599999, 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.934 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.935 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:37.936 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.937 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.897664746, 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.938 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.939 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.940 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.941 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.942 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.897217987, 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:37.943 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:37.944 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.945 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.946 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:37.947 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:37.948 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.896790745, 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.949 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.950 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.951 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.952 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.953 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.954 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.89632379599999, 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.955 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:37.956 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.957 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.958 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:37.959 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.89595525300001, 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.960 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.961 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.962 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:37.963 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.964 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:37.965 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.968 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.89553818300001, 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.969 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:37.970 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:37.971 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:37.972 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.973 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:37.974 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.892887167, 2024-09-15T21:34:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.975 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:37.976 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.977 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.978 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.979 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.980 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.89218983800001, 2024-09-15T21:34:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:37.981 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.982 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:37.983 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.984 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:37.985 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.986 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:37.987 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.988 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.608864944, 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.989 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:37.990 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.991 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.992 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.993 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.60487501399999, 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:37.994 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:37.995 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.996 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.997 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:37.998 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:37.999 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.60421875200001, 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:38.000 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.001 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.002 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:38.003 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:38.004 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.59976519400001, 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:38.005 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.006 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:38.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:38.010 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.597032439, 2024-09-15T21:34:38.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:38.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:38.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:38.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:38.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.018 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:38.018 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:38.020 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:38.021 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:38.018+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:38.053 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '19 of 30' 2024-09-15T21:34:38.058 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:34:38.120 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:38.118+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:38.134 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:38.134 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:38.134 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:38.135 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:38.145 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:38.222 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:38.220+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:38.234 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:38.298 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:38.298 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:38.298 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:38.307 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:38.323 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:38.321+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:38.385 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:38.385 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:38.418 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:38.418 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:38.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 185.95973993600001, 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:38.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:38.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.423 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 184.95385854400001, 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:38.425 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:38.426 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.427 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 45.435339147999997, 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:38.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:38.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:38.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:38.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.433 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:38.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:38.433 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:38.433 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:38.434 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:38.434 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:38.435 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:38.436 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:38.432+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:38.440 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:38.441 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:38.458 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:38.458 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:38.475 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:38.476 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:38.493 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:38.493 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:38.510 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:38.510 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:38.527 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:38.527 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:38.535 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:38.533+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:38.549 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:38.549 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:38.558 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:38.636 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:38.635+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:38.706 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:38.706 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:38.715 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:38.759 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:38.757+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:38.823 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:38.823 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:38.831 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:38.873 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:38.871+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:38.940 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:38.938+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:38.979 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:38.979 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:38.988 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:39.040 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:39.038+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:39.043 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:34:39.061 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '20 of 30' 2024-09-15T21:34:39.098 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:39.098 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:39.106 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:39.142 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:39.140+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:39.242 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:39.241+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:39.257 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:39.257 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:39.267 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:39.343 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:39.342+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:39.382 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:39.382 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:39.391 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:39.445 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:39.443+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:39.495 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:39.495 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:39.505 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:39.546 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:39.545+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:39.647 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:39.645+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:39.648 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:39.648 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:39.658 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:39.747 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:39.745+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:39.801 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:39.801 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:39.810 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:39.848 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:39.847+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:39.949 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:39.947+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:39.953 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:39.953 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:39.961 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:40.050 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:40.048+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:40.060 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '21 of 30' 2024-09-15T21:34:40.105 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:40.105 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:40.114 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:40.151 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:40.150+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:40.252 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:40.251+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:40.257 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:40.257 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:40.266 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:40.353 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:40.351+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:40.368 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:40.368 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:40.377 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:40.454 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:40.452+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:40.479 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:40.479 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:40.489 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:40.539 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:40.573 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:40.571+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:40.632 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:40.632 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:40.641 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:40.663 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:40.663 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:40.680 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:40.680 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:40.682 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:40.680+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:40.696 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:40.696 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:40.712 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:40.713 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:40.730 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:40.731 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:40.748 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:40.749 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:40.756 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:40.754+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:40.765 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:40.765 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:40.782 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:40.782 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:40.787 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:40.787 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:40.788 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:40.788 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:40.788 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:40.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:40.857 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:40.855+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 195.47996749399999, 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:40.901 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:40.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:40.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.45662264800001, 2024-09-15T21:34:40.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:40.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:40.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:40.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.914 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.35525566499999, 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.915 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:40.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:40.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:40.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:40.921 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.355071847, 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:40.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:40.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.927 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.355033002, 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:40.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.354981028, 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.934 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:40.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.3549653, 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:40.940 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:40.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:40.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.948 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.354946652, 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.949 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:40.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:40.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:40.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.35488870099999, 2024-09-15T21:34:40.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:40.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:40.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:40.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.35487087000001, 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.962 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:40.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.35478984400001, 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.968 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.970 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.35468858900001, 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.975 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.976 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:40.977 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.978 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.979 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:40.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.35464196000001, 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.981 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:40.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:40.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.987 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.89729810700001, 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:40.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:40.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:40.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.89722891700001, 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:40.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.995 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:40.996 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:40.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:40.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.896963661, 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:41.000 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:41.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:41.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:41.006 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.896889391, 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:41.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.012 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.89686048999999, 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:41.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:41.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:41.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.89684516200001, 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:41.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:41.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:41.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.024 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.89676087399999, 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:41.025 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:41.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:41.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:41.030 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:41.031 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:41.033 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:41.034 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:41.031+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:41.060 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:34:41.064 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '22 of 30' 2024-09-15T21:34:41.075 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:41.075 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:41.145 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:41.148 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:41.148 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:41.148 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:41.149 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:41.143+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:41.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:41.239 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:41.237+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:41.324 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:41.324 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:41.325 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:41.325 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:41.325 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:41.335 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:41.333+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:41.436 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:41.434+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:41.475 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:41.475 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:41.475 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:41.476 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:41.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.476 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:41.476 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:41.476 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 195.03119834899999, 2024-09-15T21:34:41.476 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:41.477 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:41.479 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:41.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.479 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.479 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:41.479 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:41.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.481 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 195.03028531699999, 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:41.483 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.484 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:41.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:41.486 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.92441704500001, 2024-09-15T21:34:41.487 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:41.488 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.489 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:41.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:41.491 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.92400477499999, 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:41.492 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.494 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.494 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:41.494 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:41.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.494 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.494 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:41.495 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:41.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.495 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.495 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.495 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:41.495 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:41.495 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.496 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.496 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.496 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:41.496 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:41.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.496 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.497 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.497 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:41.497 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:41.497 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.497 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.497 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.498 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:41.498 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:41.498 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.498 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.498 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.499 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.92355660600001, 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.500 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.501 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.501 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:41.501 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.501 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.501 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.501 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.502 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:41.502 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.502 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.502 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.502 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.502 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:41.503 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:41.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.503 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.503 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.503 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:41.503 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:41.503 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.504 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.504 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.504 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:41.504 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:41.504 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.504 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:41.505 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.506 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.922718258, 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.507 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.508 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.509 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.510 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:41.511 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.921801792, 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.512 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.513 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:41.514 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.515 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:41.516 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.47137422200001, 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.517 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.518 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.519 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.520 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.521 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.47092746300001, 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.522 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.523 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:41.524 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.525 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:41.526 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.47050022100001, 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.527 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.528 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:41.529 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.530 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.47003327199999, 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.531 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.532 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:41.533 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.534 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:41.535 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.46966472899999, 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.536 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.537 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.538 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.539 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:41.540 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.46924765899999, 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.541 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:41.542 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.543 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.544 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.545 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.466596643, 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.546 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:41.547 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.548 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.549 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.550 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.46589931400001, 2024-09-15T21:34:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.551 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.552 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.553 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.554 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:41.555 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.556 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:41.557 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:41.558 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.18257442000001, 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.559 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.560 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:41.561 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.562 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:41.563 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.17858448999999, 2024-09-15T21:34:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.564 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.565 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.566 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.567 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:41.568 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.569 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.570 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.17792822800001, 2024-09-15T21:34:41.571 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:41.572 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.573 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.574 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:41.575 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.17347466999999, 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.576 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:41.577 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.578 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.579 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.580 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.17074191500001, 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.581 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.582 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.583 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:41.584 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:41.585 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:41.585 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:41.586 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:41.587 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:41.584+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:41.686 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:41.685+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:41.734 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:41.735 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:41.735 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:41.743 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:41.788 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:41.786+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:41.850 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:41.850 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:41.850 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:41.850 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:41.850 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:41.858 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:41.900 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:41.898+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:41.988 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:41.987+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:42.006 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:42.006 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 189.54801210299999, 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.007 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.008 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.009 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:42.010 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.011 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 188.542130711, 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.012 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:42.013 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.014 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:42.015 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.016 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 49.023611314999997, 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.017 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.018 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:42.019 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.020 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.021 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.022 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:42.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:42.057 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:34:42.065 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '23 of 30' 2024-09-15T21:34:42.113 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:42.111+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:42.168 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:42.169 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:42.169 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:42.169 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:42.175 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:42.190 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:42.189+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:42.284 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:42.284 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:42.284 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:42.284 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:42.284 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:42.291 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:42.289+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:42.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:42.393 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:42.390+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:42.442 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:42.442 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:42.442 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:42.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:42.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.442 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 190.98484994200001, 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:42.445 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.446 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.447 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 105.322459545, 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:42.449 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.450 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:42.451 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 90.952425636000001, 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.452 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:42.453 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.454 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.455 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 85.223938144000002, 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:42.456 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.458 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.459 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 25.221952907999999, 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001278429, 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.460 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 5 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.461 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219037+0000", 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1600000000000001e-06 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219049+0000", 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2153999999999999e-05 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219070+0000", 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0441e-05 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219090+0000", 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0665e-05 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220231+0000", 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001140655 2024-09-15T21:34:42.463 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220313+0000", 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2354000000000005e-05 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.464 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:42.465 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:42.493 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:42.491+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:42.594 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:42.592+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:42.613 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:42.614 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:42.614 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:42.614 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:42.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:42.708 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:42.706+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:42.770 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:42.770 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:42.770 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:42.770 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:42.770 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:42.778 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:42.794 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:42.820 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:42.818+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:42.897 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:42.895+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.929 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 105.810213299, 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.930 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:34:42.931 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:42.932 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.933 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:42.934 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:42.938 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:42.946 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:42.946 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:42.963 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:42.963 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:42.980 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:42.980 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:42.996 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:42.997 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:42.997 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:42.995+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:43.013 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:43.014 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:43.031 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:43.031 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:43.048 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:43.048 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:43.054 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:43.054 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:43.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:43.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '24 of 30' 2024-09-15T21:34:43.067 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:43.067 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:43.109 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:43.107+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:43.168 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:43.169 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:43.177 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:43.218 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:43.216+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:43.299 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:43.297+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:43.320 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:43.320 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:43.330 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:43.415 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:43.413+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:43.474 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:43.474 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:43.483 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:43.501 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:43.499+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:43.602 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:43.600+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:43.626 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:43.626 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:43.635 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:43.703 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:43.701+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:43.738 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:43.738 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:43.747 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:43.805 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:43.803+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:43.890 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:43.890 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:43.899 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:43.944 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:43.942+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:44.002 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:44.002 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:44.008 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:44.004+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:44.010 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:44.068 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '25 of 30' 2024-09-15T21:34:44.071 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:34:44.109 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:44.107+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:44.153 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:44.154 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:44.162 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:44.207 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:44.206+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:44.266 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:44.266 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:44.275 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:44.309 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:44.307+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:44.377 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:44.377 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:44.385 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:44.409 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:44.407+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:44.489 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:44.490 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:44.499 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:44.543 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:44.541+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:44.606 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:44.606 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:44.611 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:44.609+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:44.615 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:44.712 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:44.711+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:44.762 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:44.762 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:44.771 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:44.815 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:44.813+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:44.878 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:44.878 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:44.887 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:44.928 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:44.926+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:45.016 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:45.014+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:45.035 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:45.035 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:45.043 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:45.056 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:34:45.072 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '26 of 30' 2024-09-15T21:34:45.078 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:45.154 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:45.154 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:45.155 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:45.153+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:45.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:45.217 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:45.215+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:45.256 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:45.285 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:45.285 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:45.302 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:45.302 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:45.316 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 192.85768388, 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:45.317 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:45.318 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:45.319 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:45.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:45.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.320 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.320 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:45.320 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:45.320 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:45.320 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.321 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 191.851802488, 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:45.322 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.323 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:45.324 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:45.325 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.326 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 52.333283092000002, 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:45.327 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.328 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:45.329 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.330 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:45.331 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:45.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:45.331 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.331 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.331 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.331 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.331 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.331 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:45.332 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:45.315+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:45.332 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:45.332 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:45.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:45.338 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:45.338 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:45.355 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:45.356 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:45.373 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:45.373 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:45.390 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:45.390 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:45.418 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:45.416+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:45.442 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:45.443 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:45.452 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:45.536 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:45.534+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:45.600 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:45.600 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:45.601 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:45.601 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:45.601 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:45.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:45.651 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:45.649+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:45.721 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:45.719+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 194.300522755, 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:45.758 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:45.759 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.760 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:45.761 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:45.762 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.63813235800001, 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:45.763 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.764 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:45.765 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.766 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 94.268098449000007, 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:45.767 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.768 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.769 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:45.770 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 88.539610956999994, 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:45.771 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.775 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 28.537625721000001, 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001278429, 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:45.776 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 5 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219037+0000", 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1600000000000001e-06 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219049+0000", 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2153999999999999e-05 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219070+0000", 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0441e-05 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219090+0000", 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0665e-05 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220231+0000", 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001140655 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:45.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:45.780 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220313+0000", 2024-09-15T21:34:45.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2354000000000005e-05 2024-09-15T21:34:45.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.780 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.780 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:45.780 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:45.780 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:45.781 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:45.822 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:45.820+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:45.923 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:45.921+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:45.929 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:45.929 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:45.929 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:45.929 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:45.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:46.023 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:46.022+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:46.045 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:46.045 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:46.045 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:46.054 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:46.072 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '27 of 30' 2024-09-15T21:34:46.076 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:46.076 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:46.076 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-15T21:34:46.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T21:34:46.124 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:46.122+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:46.165 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:46.165 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 109.04623254000001, 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.166 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:34:46.167 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:46.168 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.169 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:46.170 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:34:46.170 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:34:46.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.170 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.170 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.170 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.170 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:46.175 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:46.225 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:46.223+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:46.291 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:46.291 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:46.291 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:46.291 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:46.300 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:46.344 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:46.342+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:46.403 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:46.403 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:46.404 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:46.404 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:46.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:46.425 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:34:46.425 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:46.425 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 73, 2024-09-15T21:34:46.425 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2024-09-15T21:31:12.329884+0000", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2024-09-15T21:34:45.051437+0000", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "last_up_change": "2024-09-15T21:31:25.414443+0000", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "last_in_change": "2024-09-15T21:33:28.789828+0000", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_num": 5806080, 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_set": [ 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "nodeep-scrub", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "noscrub", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "pglog_hardlimit", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "purged_snapdirs", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_deletes", 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "sortbitwise" 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:34:46.426 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_version": 5, 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_snapshot": "", 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_max": 2, 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd": 8, 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "require_min_compat_client": "luminous", 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "min_compat_client": "jewel", 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "require_osd_release": "squid", 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_crimson": false, 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "pools": [ 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.427 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 1, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": ".mgr", 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:22.622713+0000", 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool", 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:34:46.428 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 1, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 1, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 1, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 1, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 1, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:34:46.429 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "18", 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 0, 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 0, 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:34:46.430 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:34:46.431 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:34:46.432 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "options": { 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_max": 32, 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_min": 1 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": {} 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 6.9000000953674316, 2024-09-15T21:34:46.433 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 6.9000000953674316, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 0.28999999165534973, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 2, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 2, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 0.85879135131835938, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 2, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": "rbd", 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:30.313202+0000", 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 8193, 2024-09-15T21:34:46.434 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 8, 2024-09-15T21:34:46.435 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 8, 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 8, 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 8, 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 8, 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "22", 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:34:46.436 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 2, 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 22, 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:34:46.437 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:34:46.438 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "options": {}, 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd": {} 2024-09-15T21:34:46.439 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 2.630000114440918, 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 2.630000114440918, 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 1, 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 2.630000114440918, 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 2.630000114440918, 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 0.85879135131835938, 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.440 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "osds": [ 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "f7eb9035-e8c2-4d90-a198-2bed78d5a3b5", 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 66, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:34:46.441 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6824", 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6825", 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.442 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6826", 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6827", 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.443 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6830", 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6831", 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:34:46.444 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6828", 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6829", 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:34:46.445 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6825/368705522", 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6827/368705522", 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6831/368705522", 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6829/368705522", 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.446 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a", 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 66, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:34:46.447 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6816", 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6817", 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.448 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6818", 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6819", 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.449 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6822", 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6823", 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.450 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6820", 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6821", 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:34:46.451 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6817/3888640163", 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6819/3888640163", 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6823/3888640163", 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6821/3888640163", 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.452 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "180bae67-0f64-45a0-9a58-f4b8ff2b90a5", 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 36, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.453 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6808", 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6809", 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:34:46.454 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6810", 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6811", 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:34:46.455 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6814", 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6815", 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.456 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6812", 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6813", 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.457 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6809/1831747042", 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6811/1831747042", 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6815/1831747042", 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6813/1831747042", 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d05cec44-e228-4297-98c8-53e2ad91b37c", 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:34:46.458 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-09-15T21:34:46.459 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.460 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6802", 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6803", 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.461 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6806", 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6807", 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.462 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6804", 2024-09-15T21:34:46.463 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:34:46.464 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.464 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.464 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6805", 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6801/3658047863", 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6803/3658047863", 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6807/3658047863", 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6805/3658047863", 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:34:46.465 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "5d7097a2-8165-4885-a356-f8e67851be5d", 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 0.870330810546875, 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:34:46.466 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6816", 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.467 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6817", 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6818", 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:34:46.468 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6819", 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6822", 2024-09-15T21:34:46.469 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6823", 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6820", 2024-09-15T21:34:46.470 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6821", 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6817/1988554297", 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6819/1988554297", 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1988554297", 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1988554297", 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:34:46.471 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f", 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 14, 2024-09-15T21:34:46.472 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 52, 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6824", 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6825", 2024-09-15T21:34:46.473 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6826", 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6827", 2024-09-15T21:34:46.474 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6830", 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6831", 2024-09-15T21:34:46.475 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6828", 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.476 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6829", 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6825/1071217311", 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6827/1071217311", 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6831/1071217311", 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6829/1071217311", 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.477 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612", 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 43, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:34:46.478 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6800", 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6801", 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:34:46.479 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6802", 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6803", 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:34:46.480 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6806", 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6807", 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:34:46.481 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6804", 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6805", 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.482 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6801/1069316293", 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6803/1069316293", 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6807/1069316293", 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6805/1069316293", 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "fb77f072-c3ae-4914-81bd-473d30016320", 2024-09-15T21:34:46.483 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 0, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 43, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.484 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6808", 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6809", 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.485 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6810", 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6811", 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.486 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6814", 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6815", 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.487 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6812", 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6813", 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6809/3119270077", 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6811/3119270077", 2024-09-15T21:34:46.488 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6815/3119270077", 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6813/3119270077", 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_xinfo": [ 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:34:46.489 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.795411+0000", 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.139237+0000", 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:34:46.490 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.562583+0000", 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:34:46.491 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.363254+0000", 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:34:46.492 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.344635+0000", 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.323050+0000", 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:34:46.493 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.381780+0000", 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:34:46.494 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.049941+0000", 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap": [], 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_items": [], 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_primaries": [], 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_temp": [], 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_temp": [], 2024-09-15T21:34:46.495 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklist": {}, 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "range_blocklist": {}, 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profiles": { 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "default": { 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-failure-domain": "osd", 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "k": "2", 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "m": "1", 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "plugin": "jerasure", 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "technique": "reed_sol_van" 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps_queue": [], 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "new_removed_snaps": [], 2024-09-15T21:34:46.496 INFO:teuthology.orchestra.run.smithi019.stdout: "new_purged_snaps": [], 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_node_flags": {}, 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: "device_class_flags": {}, 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": { 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_enabled": false, 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_bucket_count": 0, 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_bucket": 0 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:46.497 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:46.499 INFO:tasks.thrashosds.thrasher:j is {'epoch': 73, 'fsid': '13019d66-d4b6-4e2a-8322-38bcb4210bcf', 'created': '2024-09-15T21:31:12.329884+0000', 'modified': '2024-09-15T21:34:45.051437+0000', 'last_up_change': '2024-09-15T21:31:25.414443+0000', 'last_in_change': '2024-09-15T21:33:28.789828+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T21:31:22.622713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8587913513183594, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:31:30.313202+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.8587913513183594, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}], 'osds': [{'osd': 0, 'uuid': 'f7eb9035-e8c2-4d90-a198-2bed78d5a3b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6824', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6825', 'nonce': 368705522}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6826', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6827', 'nonce': 368705522}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6830', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6831', 'nonce': 368705522}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6828', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6829', 'nonce': 368705522}]}, 'public_addr': '172.21.15.19:6825/368705522', 'cluster_addr': '172.21.15.19:6827/368705522', 'heartbeat_back_addr': '172.21.15.19:6831/368705522', 'heartbeat_front_addr': '172.21.15.19:6829/368705522', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd2bcecd4-0bb0-4b38-b395-8b6dbb48f53a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6816', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6817', 'nonce': 3888640163}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6818', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6819', 'nonce': 3888640163}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6822', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6823', 'nonce': 3888640163}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6820', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6821', 'nonce': 3888640163}]}, 'public_addr': '172.21.15.19:6817/3888640163', 'cluster_addr': '172.21.15.19:6819/3888640163', 'heartbeat_back_addr': '172.21.15.19:6823/3888640163', 'heartbeat_front_addr': '172.21.15.19:6821/3888640163', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '180bae67-0f64-45a0-9a58-f4b8ff2b90a5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6808', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6809', 'nonce': 1831747042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6810', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6811', 'nonce': 1831747042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6814', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6815', 'nonce': 1831747042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6812', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6813', 'nonce': 1831747042}]}, 'public_addr': '172.21.15.19:6809/1831747042', 'cluster_addr': '172.21.15.19:6811/1831747042', 'heartbeat_back_addr': '172.21.15.19:6815/1831747042', 'heartbeat_front_addr': '172.21.15.19:6813/1831747042', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd05cec44-e228-4297-98c8-53e2ad91b37c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6800', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6801', 'nonce': 3658047863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6802', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6803', 'nonce': 3658047863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6806', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6807', 'nonce': 3658047863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6804', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6805', 'nonce': 3658047863}]}, 'public_addr': '172.21.15.19:6801/3658047863', 'cluster_addr': '172.21.15.19:6803/3658047863', 'heartbeat_back_addr': '172.21.15.19:6807/3658047863', 'heartbeat_front_addr': '172.21.15.19:6805/3658047863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '5d7097a2-8165-4885-a356-f8e67851be5d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.870330810546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1988554297}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1988554297}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1988554297}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1988554297}]}, 'public_addr': '172.21.15.192:6817/1988554297', 'cluster_addr': '172.21.15.192:6819/1988554297', 'heartbeat_back_addr': '172.21.15.192:6823/1988554297', 'heartbeat_front_addr': '172.21.15.192:6821/1988554297', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 1071217311}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 1071217311}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 1071217311}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 1071217311}]}, 'public_addr': '172.21.15.192:6825/1071217311', 'cluster_addr': '172.21.15.192:6827/1071217311', 'heartbeat_back_addr': '172.21.15.192:6831/1071217311', 'heartbeat_front_addr': '172.21.15.192:6829/1071217311', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 1069316293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 1069316293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 1069316293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 1069316293}]}, 'public_addr': '172.21.15.192:6801/1069316293', 'cluster_addr': '172.21.15.192:6803/1069316293', 'heartbeat_back_addr': '172.21.15.192:6807/1069316293', 'heartbeat_front_addr': '172.21.15.192:6805/1069316293', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'fb77f072-c3ae-4914-81bd-473d30016320', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 3119270077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 3119270077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 3119270077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 3119270077}]}, 'public_addr': '172.21.15.192:6809/3119270077', 'cluster_addr': '172.21.15.192:6811/3119270077', 'heartbeat_back_addr': '172.21.15.192:6815/3119270077', 'heartbeat_front_addr': '172.21.15.192:6813/3119270077', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.795411+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.139237+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.562583+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.363254+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.344635+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.323050+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.381780+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.049941+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T21:34:46.500 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:46.500 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:46.496+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:46.600 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:46.598+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:46.600 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:46.600 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:46.600 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:46.601 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:34:46.601 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:46.609 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:46.701 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:46.699+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:46.712 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:46.712 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:46.712 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:46.721 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:46.802 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:46.800+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:46.834 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:34:46.835 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:34:46.847 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-15T21:34:46.643079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":528966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":222848,"kb_used_data":6620,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751872,"statfs":{"total":773094113280,"available":772865916928,"internally_reserved":0,"allocated":6778880,"data_stored":3865250,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.600098"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":131,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T21:34:42.055211+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:34:42.055211+0000","last_peered":"2024-09-15T21:34:42.055211+0000","last_clean":"2024-09-15T21:34:42.055211+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:34:42.055211+0000","last_undegraded":"2024-09-15T21:34:42.055211+0000","last_fullsized":"2024-09-15T21:34:42.055211+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-15T21:34:45.056924+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:34:45.056924+0000","last_peered":"2024-09-15T21:34:45.056924+0000","last_clean":"2024-09-15T21:34:45.056924+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:34:45.056924+0000","last_undegraded":"2024-09-15T21:34:45.056924+0000","last_fullsized":"2024-09-15T21:34:45.056924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-15T21:34:45.163496+0000","last_change":"2024-09-15T21:34:32.071755+0000","last_active":"2024-09-15T21:34:45.163496+0000","last_peered":"2024-09-15T21:34:45.163496+0000","last_clean":"2024-09-15T21:34:45.163496+0000","last_became_active":"2024-09-15T21:34:32.071516+0000","last_became_peered":"2024-09-15T21:34:32.071516+0000","last_unstale":"2024-09-15T21:34:45.163496+0000","last_undegraded":"2024-09-15T21:34:45.163496+0000","last_fullsized":"2024-09-15T21:34:45.163496+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:37:56.220605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":120,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-15T21:34:45.056924+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:34:45.056924+0000","last_peered":"2024-09-15T21:34:45.056924+0000","last_clean":"2024-09-15T21:34:45.056924+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:34:45.056924+0000","last_undegraded":"2024-09-15T21:34:45.056924+0000","last_fullsized":"2024-09-15T21:34:45.056924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":85,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T21:34:42.370545+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:34:42.370545+0000","last_peered":"2024-09-15T21:34:42.370545+0000","last_clean":"2024-09-15T21:34:42.370545+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:34:42.370545+0000","last_undegraded":"2024-09-15T21:34:42.370545+0000","last_fullsized":"2024-09-15T21:34:42.370545+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":118,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T21:34:42.054682+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:34:42.054682+0000","last_peered":"2024-09-15T21:34:42.054682+0000","last_clean":"2024-09-15T21:34:42.054682+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:34:42.054682+0000","last_undegraded":"2024-09-15T21:34:42.054682+0000","last_fullsized":"2024-09-15T21:34:42.054682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":63,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-15T21:34:45.056924+0000","last_change":"2024-09-15T21:34:32.071917+0000","last_active":"2024-09-15T21:34:45.056924+0000","last_peered":"2024-09-15T21:34:45.056924+0000","last_clean":"2024-09-15T21:34:45.056924+0000","last_became_active":"2024-09-15T21:34:32.071605+0000","last_became_peered":"2024-09-15T21:34:32.071605+0000","last_unstale":"2024-09-15T21:34:45.056924+0000","last_undegraded":"2024-09-15T21:34:45.056924+0000","last_fullsized":"2024-09-15T21:34:45.056924+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:46.837069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":75,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-15T21:34:42.370531+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:34:42.370531+0000","last_peered":"2024-09-15T21:34:42.370531+0000","last_clean":"2024-09-15T21:34:42.370531+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:34:42.370531+0000","last_undegraded":"2024-09-15T21:34:42.370531+0000","last_fullsized":"2024-09-15T21:34:42.370531+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":165,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-15T21:34:45.163496+0000","last_change":"2024-09-15T21:34:32.072376+0000","last_active":"2024-09-15T21:34:45.163496+0000","last_peered":"2024-09-15T21:34:45.163496+0000","last_clean":"2024-09-15T21:34:45.163496+0000","last_became_active":"2024-09-15T21:34:32.072003+0000","last_became_peered":"2024-09-15T21:34:32.072003+0000","last_unstale":"2024-09-15T21:34:45.163496+0000","last_undegraded":"2024-09-15T21:34:45.163496+0000","last_fullsized":"2024-09-15T21:34:45.163496+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:03.086225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476780,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343948,"statfs":{"total":96636764160,"available":96608202752,"internally_reserved":0,"allocated":884736,"data_stored":883929,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476782,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":349559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476781,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":355020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476781,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27904,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343936,"statfs":{"total":96636764160,"available":96608190464,"internally_reserved":0,"allocated":897024,"data_stored":883929,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27844,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343996,"statfs":{"total":96636764160,"available":96608251904,"internally_reserved":0,"allocated":835584,"data_stored":349578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574891,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":349559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":819200,"data_stored":344117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":349559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:46.848 INFO:tasks.thrashosds.thrasher:Setting 2.5 to [6, 4] 2024-09-15T21:34:46.848 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.5', '6', '4'] 2024-09-15T21:34:46.848 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.5 6 4 2024-09-15T21:34:46.864 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:46.864 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:46.865 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:46.865 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:46.873 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:46.932 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:46.930+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:46.978 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.978 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 195.52337836999999, 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.979 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:46.980 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.981 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:46.982 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:46.983 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:46.984 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.985 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 195.51723874000001, 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.986 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:46.987 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.988 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:46.989 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 195.51407056400001, 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:46.990 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:46.991 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:46.992 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:46.993 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:46.994 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 195.513524811, 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:46.995 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:46.996 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:46.997 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.998 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:46.999 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.52088650300001, 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:47.000 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.001 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.002 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.003 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.004 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.005 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.006 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:47.007 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 194.517593675, 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.008 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.009 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.010 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.011 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.515042905, 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.012 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.013 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.014 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:47.015 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.016 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:47.017 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.018 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:47.019 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 193.51164221600001, 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.020 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.021 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:47.022 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:47.023 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.487407078000004, 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.024 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.025 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.026 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.027 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.028 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.029 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:47.030 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.484400913000002, 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.031 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.032 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:47.033 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.034 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 24.06142646, 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023751800000000001, 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.035 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.036 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914744+0000", 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5999999999999999e-06 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914750+0000", 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914759+0000", 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2e-06 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.037 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914768+0000", 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4689999999999997e-06 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914858+0000", 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0499000000000002e-05 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914980+0000", 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012141099999999999 2024-09-15T21:34:47.038 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.039 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.039 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:47.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:47.041 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:47.038+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:47.072 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:34:47.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '28 of 30' 2024-09-15T21:34:47.141 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:47.139+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:47.185 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:47.185 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:47.185 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:47.185 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:47.193 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:47.278 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:47.276+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:47.336 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:47.336 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:47.337 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:47.337 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:47.337 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:47.343 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:47.341+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:47.345 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:47.403 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:47.447 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:47.445+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:47.489 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:47.489 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 144.57491073099999, 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:47.490 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.491 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:47.492 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.493 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 136.548573312, 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:47.494 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.495 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.496 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:47.497 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 136.54514004399999, 2024-09-15T21:34:47.498 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.499 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.500 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:47.501 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.502 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 95.994538464000001, 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.503 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.504 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.505 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.506 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 84.574955840000001, 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.507 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:47.508 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.509 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:47.510 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:47.511 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:47.511 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:47.512 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:47.528 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:47.528 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:47.545 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:47.545 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:47.545 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:47.543+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:47.561 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:47.561 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:47.577 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:47.577 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:47.594 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:47.594 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:47.610 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:47.610 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:47.616 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:47.616 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:47.617 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:47.617 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:47.625 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:47.626 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:47.627 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:47.643 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:47.643 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:47.645 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:47.644+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:set 2.5 pg_upmap mapping to [6,4] 2024-09-15T21:34:47.722 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:34:47.730 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:47.730 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:47.739 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:47.784 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:47.782+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:47.842 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:47.842 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:47.847 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:47.846+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:47.851 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:47.948 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:47.947+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:47.959 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:47.959 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:47.968 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:48.050 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:48.048+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:48.076 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '29 of 30' 2024-09-15T21:34:48.118 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:48.118 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:48.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:48.151 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:48.149+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:48.251 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:48.250+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:48.277 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:48.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:48.371 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:48.369+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:48.434 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:48.435 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:48.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:48.485 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:48.483+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:48.555 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:48.553+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:48.591 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:48.591 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:48.599 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:48.656 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:48.654+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:48.707 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:48.716 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:48.757 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:48.756+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:48.858 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:48.857+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:48.864 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:48.864 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:48.873 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:48.959 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:48.958+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:48.980 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:48.981 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:48.989 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:49.074 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:49.072+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:49.082 INFO:tasks.mon_recovery:adding mon c back in 2024-09-15T21:34:49.082 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:34:49.082 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:34:49.126 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:34:49.126 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-09-15T21:34:49.127 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:49.143 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:49.143 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:49.154 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:49.195 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:49.193+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:49.263 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:49.261+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:49.304 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:49.305 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:49.315 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:49.399 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:49.397+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:49.464 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:49.464 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:49.464 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:49.462+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:49.472 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:49.506 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":62,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":29,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:49.565 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:49.563+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:49.613 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:49.614 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:49.622 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:49.654 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:49.666 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:49.664+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:49.738 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:49.738 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:49.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:34:49.747 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:49.785 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:49.783+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:49.806 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:49.806 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:49.840 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:49.840 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:49.851 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:49.851 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:49.857 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:49.857 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:49.861 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:49.867 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:49.865+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:49.874 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:49.874 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:49.890 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:49.891 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:49.908 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:49.908 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:49.924 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:49.925 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:49.973 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:49.994 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:49.992+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:50.069 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:50.067+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:50.076 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:50.076 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:50.076 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:50.076 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:50.086 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:50.171 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:50.169+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:50.230 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:50.230 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:50.230 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:50.230 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:50.230 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:50.238 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:50.283 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:50.281+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:50.342 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:50.342 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 198.887070139, 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.343 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:34:50.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:34:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:34:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:34:50.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:34:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:34:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:34:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:50.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:34:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:34:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:50.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:34:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:34:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:34:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:34:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 198.880930509, 2024-09-15T21:34:50.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:50.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:34:50.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.358 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 198.87776233299999, 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:50.359 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:34:50.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:34:50.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:34:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:34:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 198.87721658000001, 2024-09-15T21:34:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:34:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:50.364 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:50.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.366 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:34:50.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 197.884578272, 2024-09-15T21:34:50.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:50.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.371 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.372 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:34:50.374 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:34:50.375 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.376 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 197.88128544400001, 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:50.377 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.378 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:34:50.379 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.380 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 196.87873467399999, 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:50.381 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:34:50.382 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:34:50.383 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:34:50.384 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:34:50.385 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:34:50.386 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:34:50.387 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.388 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 196.875333985, 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:50.389 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.390 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.391 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:34:50.392 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.851098847000003, 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.393 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.394 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:34:50.395 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:34:50.396 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.848092682000001, 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:50.397 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.398 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:34:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-15T21:34:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 27.425118228999999, 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023751800000000001, 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.402 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914744+0000", 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5999999999999999e-06 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.403 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914750+0000", 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914759+0000", 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2e-06 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914768+0000", 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4689999999999997e-06 2024-09-15T21:34:50.404 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914858+0000", 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0499000000000002e-05 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914980+0000", 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012141099999999999 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.405 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.406 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.406 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:50.407 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:50.408 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:50.406+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:50.508 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:50.506+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:50.550 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:50.550 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:50.550 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:50.550 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:50.559 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:50.609 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:50.607+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:50.702 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:50.703 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:50.703 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:50.703 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:50.703 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:50.709 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:50.708+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:50.711 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:50.810 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:50.808+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 147.90083827800001, 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:34:50.815 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:50.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.819 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 139.87450085899999, 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:34:50.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 139.87106759100001, 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.824 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:34:50.828 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 99.320466010999994, 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:34:50.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:50.832 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 87.900883386999993, 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:50.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:50.836 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:50.837 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:50.837 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:50.911 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:50.909+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:50.980 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:50.981 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:50.981 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:50.989 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:51.033 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:51.031+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:51.075 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:34:51.093 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:51.093 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:51.093 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:51.093 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:51.093 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:51.102 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:51.143 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:51.141+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:51.213 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:51.211+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 205.825040331, 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.246 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.247 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.248 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.249 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.250 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.251 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:51.252 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.253 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.80169548500001, 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.254 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.255 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:51.256 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.257 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.258 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.259 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.70032850199999, 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.260 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.261 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.262 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.263 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.264 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.265 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.266 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.70014468400001, 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.267 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:51.268 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.269 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.270 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.271 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.272 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.700105839, 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.273 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.274 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.275 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.276 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:51.277 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:51.278 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.700053865, 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.279 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:51.280 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.281 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:51.282 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.283 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.284 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.70003813700001, 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.285 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.286 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.287 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:51.288 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.289 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.290 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.700019489, 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.291 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.292 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:51.293 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.294 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:51.295 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:51.296 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:51.297 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.699961538, 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:51.298 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.299 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.300 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:51.301 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.302 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.303 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.69994370699999, 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.304 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:51.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:51.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:51.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.310 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.69986268100001, 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:51.312 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:51.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.316 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.69976142600001, 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:51.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:51.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:51.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.69971479700001, 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.323 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.324 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:51.325 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.326 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:51.327 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.328 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.24237094399999, 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.329 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.330 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.331 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.332 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:51.333 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.334 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.24230175400001, 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.335 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.336 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:51.337 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:51.338 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.339 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:51.340 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.242036498, 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:51.341 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.342 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:51.343 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:51.344 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.345 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.346 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.24196222800001, 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:51.347 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:51.349 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.350 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:51.351 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.352 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:51.353 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.241933327, 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.354 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.355 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:51.356 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.357 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.358 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.359 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.24191799900001, 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.360 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:51.361 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:51.362 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.363 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:51.364 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.365 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 204.241833711, 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:51.366 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:51.367 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.368 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:51.369 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:51.370 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:51.373 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:51.374 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:51.376 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:51.376 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:51.374+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:51.477 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:51.475+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:51.496 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:51.497 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:51.497 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:51.497 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:51.506 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:51.580 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:51.575+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:51.614 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:51.614 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:51.614 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:51.614 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:51.614 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:51.622 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:51.678 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:51.676+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:51.772 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:51.772 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:51.772 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:51.772 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:51.772 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.772 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:51.772 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 205.328591875, 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.773 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:51.774 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:51.775 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.776 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.777 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 205.327678843, 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.778 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.779 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:51.780 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:51.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.781 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.781 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.781 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:51.781 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:51.781 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:51.782 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 205.22181057099999, 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.783 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.784 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.784 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.784 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.785 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.786 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:51.787 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:51.788 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.789 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.789 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.789 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.789 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.789 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:51.789 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:51.789 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 205.22139830099999, 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.790 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:51.791 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.792 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:51.793 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:51.794 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 205.22095013200001, 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.795 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:51.796 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.797 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:51.798 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 205.22011178400001, 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:51.799 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.800 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.801 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.802 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:51.803 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 205.219195318, 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.804 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.805 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.806 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:51.807 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.808 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.76876774799999, 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.809 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.810 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.811 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:51.812 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.813 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.76832098899999, 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:51.814 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.815 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:51.816 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.817 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.818 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.76789374699999, 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.819 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.820 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.821 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:51.822 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.767426798, 2024-09-15T21:34:51.823 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.824 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:51.825 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.826 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.827 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:51.828 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.76705825499999, 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.829 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.830 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:51.831 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.832 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.766641185, 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.833 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.834 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:51.835 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.836 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.837 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.76399016900001, 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.838 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.839 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:51.840 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.841 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.842 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.76329283999999, 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.843 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.844 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:51.845 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.846 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:51.847 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.47996794599999, 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.848 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:51.849 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.850 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.851 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.851 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:51.851 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:51.851 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.852 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.475978016, 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.853 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.854 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.855 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.856 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.857 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.47532175399999, 2024-09-15T21:34:51.858 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:51.859 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.860 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.861 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:51.862 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:51.863 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.470868196, 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.864 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.865 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.865 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.865 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:51.865 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.865 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.865 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:51.866 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:51.867 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.868 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.46813544099999, 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:51.869 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:51.870 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.871 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:51.872 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:51.873 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:51.875 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:51.875 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:51.872+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:51.953 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:51.976 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:51.974+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:52.023 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:52.024 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:52.024 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:52.024 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:52.033 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:52.078 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:52.076+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:52.103 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:52.103 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:52.136 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:52.137 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:52.170 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:52.170 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:52.177 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:52.175+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:52.191 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:52.191 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:52.198 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:52.198 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:52.201 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:52.216 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:52.217 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:52.234 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:52.234 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:52.251 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:52.251 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:52.269 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:52.269 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:52.278 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:52.276+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:52.325 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:52.325 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:52.333 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:52.378 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:52.376+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:52.441 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:52.441 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:52.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:52.492 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:52.490+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:52.520 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:52.581 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:52.579+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:52.600 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:52.600 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:52.609 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:52.682 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:52.680+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:52.741 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:52.741 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:52.758 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:52.758 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:52.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:52.811 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:52.809+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:52.875 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:52.875 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:52.884 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:52.884 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:52.882+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:52.907 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":62,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":32,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:52.984 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:52.982+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:53.014 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:53.014 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:53.023 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:53.092 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-15T21:34:53.107 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:53.105+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:53.171 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:53.171 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:53.181 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:53.225 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:53.223+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:53.287 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:53.285+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:53.297 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:53.297 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:53.306 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:53.390 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:53.388+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:53.450 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:53.450 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:53.459 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:53.489 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:53.487+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:53.590 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:53.588+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:53.601 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:53.602 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:53.610 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:53.691 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:53.689+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:53.712 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:53.712 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:53.721 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:53.792 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:53.789+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:53.824 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:53.824 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:53.832 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:53.917 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:53.915+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:53.976 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:53.976 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:53.985 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:53.994 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:53.993+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:54.081 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is set 2024-09-15T21:34:54.095 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:54.093+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:54.130 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:54.130 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:54.138 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:54.223 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:54.221+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:54.280 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:54.281 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:54.282 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:54.290 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:54.326 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:54.324+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:54.397 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:54.396+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:54.419 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:54.420 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:54.434 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:54.434 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:54.437 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:54.437 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:54.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:54.453 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:54.453 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:54.469 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:54.469 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:54.485 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:54.486 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:54.498 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:54.496+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:54.502 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:54.503 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:54.519 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:54.520 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:54.537 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:54.537 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:54.548 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:54.549 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:54.549 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:34:54.549 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:54.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:54.600 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:54.598+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:54.659 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:34:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:34:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:34:54.659 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:54.668 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:54.701 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:54.699+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:54.801 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:54.800+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 209.39194830700001, 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.007783256, 2024-09-15T21:34:54.813 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.814 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417876+0000", 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417878+0000", 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.7740000000000001e-06 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417887+0000", 2024-09-15T21:34:54.815 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4679999999999992e-06 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417900+0000", 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.417909+0000", 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.6070000000000008e-06 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.816 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418931+0000", 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001021812 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:25.418944+0000", 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2948999999999999e-05 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424291+0000", 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0053477319999999 2024-09-15T21:34:54.817 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424427+0000", 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000135919 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.424582+0000", 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015443100000000001 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.818 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425334+0000", 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000752395 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425570+0000", 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023604299999999999 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425610+0000", 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0342000000000001e-05 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.819 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.425660+0000", 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9376000000000003e-05 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:25 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:54.820 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.36860346099999, 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0090767190000000005, 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 25 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.821 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441221+0000", 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441222+0000", 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8500000000000002e-07 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.822 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441226+0000", 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7950000000000001e-06 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441234+0000", 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3489999999999992e-06 2024-09-15T21:34:54.823 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.441251+0000", 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6937999999999999e-05 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445372+0000", 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0041206350000000001 2024-09-15T21:34:54.824 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445454+0000", 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2065999999999997e-05 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.445581+0000", 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012789400000000001 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.825 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.446408+0000", 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00082691399999999999 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450236+0000", 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003827577 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450272+0000", 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5806999999999998e-05 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.826 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.450298+0000", 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6259e-05 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:42 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.267236478, 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073191239999999998, 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.827 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 42 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.828 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542588+0000", 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542589+0000", 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8599999999999999e-07 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.829 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542591+0000", 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4310000000000001e-06 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542595+0000", 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8840000000000004e-06 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542599+0000", 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.54e-06 2024-09-15T21:34:54.830 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544142+0000", 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015438509999999999 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544191+0000", 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8636e-05 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544275+0000", 2024-09-15T21:34:54.831 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4257000000000003e-05 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545436+0000", 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011602299999999999 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549838+0000", 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044025100000000001 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.832 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549865+0000", 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7172e-05 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549907+0000", 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2227000000000001e-05 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:43 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.833 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.26705265999999, 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072049130000000003, 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 43 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.834 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542772+0000", 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542773+0000", 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0380000000000001e-06 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.835 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542779+0000", 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8470000000000001e-06 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542789+0000", 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0345e-05 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542795+0000", 2024-09-15T21:34:54.836 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.7060000000000001e-06 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544819+0000", 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002023567 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.544897+0000", 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8578999999999994e-05 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.837 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545031+0000", 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00013392499999999999 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546914+0000", 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001882583 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549916+0000", 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030023670000000001 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.838 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549944+0000", 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7854000000000001e-05 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549977+0000", 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3102000000000003e-05 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.839 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:44 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.26701381500001, 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072370129999999996, 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 44 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.840 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542811+0000", 2024-09-15T21:34:54.841 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9900000000000001e-07 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542845+0000", 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3472000000000001e-05 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542851+0000", 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.5690000000000001e-06 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.842 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542855+0000", 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8489999999999999e-06 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545477+0000", 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026213959999999998 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.843 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545551+0000", 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4836e-05 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.545677+0000", 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000125819 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548014+0000", 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0023362510000000001 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.844 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549983+0000", 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001969494 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550019+0000", 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6307000000000003e-05 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550048+0000", 2024-09-15T21:34:54.845 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8620999999999999e-05 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:45 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.26696184100001, 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0072425529999999997, 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.846 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 45 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:54.847 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542863+0000", 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6600000000000002e-07 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542867+0000", 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.642e-06 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.848 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542871+0000", 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6160000000000002e-06 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542874+0000", 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546003+0000", 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031290799999999998 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.849 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546085+0000", 2024-09-15T21:34:54.850 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2748000000000001e-05 2024-09-15T21:34:54.850 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.850 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.850 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.850 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546212+0000", 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012632600000000001 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548082+0000", 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001870586 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550053+0000", 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0019704369999999998 2024-09-15T21:34:54.852 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550082+0000", 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.8926999999999999e-05 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550106+0000", 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3879e-05 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.853 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:46 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.26694611299999, 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00728322, 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 46 2024-09-15T21:34:54.854 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.855 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542879+0000", 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.72e-07 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542883+0000", 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.557e-06 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542887+0000", 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6380000000000003e-06 2024-09-15T21:34:54.856 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542890+0000", 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0699999999999998e-06 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546531+0000", 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0036409319999999999 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.857 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546584+0000", 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2370999999999997e-05 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546663+0000", 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9711999999999998e-05 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549784+0000", 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031207280000000001 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.858 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550112+0000", 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00032755399999999997 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550142+0000", 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.074e-05 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550162+0000", 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9646000000000001e-05 2024-09-15T21:34:54.859 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:47 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.26692746500001, 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0073232169999999999, 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.860 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 47 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542897+0000", 2024-09-15T21:34:54.861 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542898+0000", 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542936+0000", 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8608e-05 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.862 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542944+0000", 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6410000000000002e-06 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542947+0000", 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5939999999999998e-06 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546951+0000", 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0040036849999999999 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.863 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.546996+0000", 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4388999999999998e-05 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547156+0000", 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000160173 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549830+0000", 2024-09-15T21:34:54.864 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002674339 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550167+0000", 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00033710899999999999 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550200+0000", 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3188000000000001e-05 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.865 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550221+0000", 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0171999999999999e-05 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:48 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.26686951400001, 2024-09-15T21:34:54.866 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.007778025, 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 48 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:54.867 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542955+0000", 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542956+0000", 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6100000000000002e-07 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.868 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542959+0000", 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.0290000000000001e-06 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542962+0000", 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4800000000000001e-06 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542966+0000", 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1810000000000003e-06 2024-09-15T21:34:54.869 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547544+0000", 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0045780559999999996 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547599+0000", 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4908999999999999e-05 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.870 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.547678+0000", 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9068999999999998e-05 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550226+0000", 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025474590000000002 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550685+0000", 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000459421 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.871 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550712+0000", 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6740000000000001e-05 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550733+0000", 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.1319999999999999e-05 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.872 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:49 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.266851683, 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010147252000000001, 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 49 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.873 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542973+0000", 2024-09-15T21:34:54.874 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5400000000000002e-07 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542976+0000", 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4549999999999998e-06 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542980+0000", 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.7400000000000002e-06 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.875 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.542983+0000", 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.631e-06 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548107+0000", 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051232120000000003 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548153+0000", 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6007000000000002e-05 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.876 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548228+0000", 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.5969000000000004e-05 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550231+0000", 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020020429999999998 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553053+0000", 2024-09-15T21:34:54.877 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0028229629999999999 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553093+0000", 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.9811000000000002e-05 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553120+0000", 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7067e-05 2024-09-15T21:34:54.878 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:50 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.266770657, 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010211380000000001, 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.879 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 50 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.880 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543054+0000", 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543055+0000", 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7199999999999998e-07 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543105+0000", 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.9867000000000002e-05 2024-09-15T21:34:54.881 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543113+0000", 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1159999999999999e-06 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543128+0000", 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5051999999999999e-05 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548464+0000", 2024-09-15T21:34:54.882 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0053358659999999999 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548543+0000", 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9679999999999996e-05 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.548694+0000", 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015015699999999999 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.883 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552260+0000", 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035663090000000001 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553173+0000", 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00091287599999999997 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553225+0000", 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2049000000000003e-05 2024-09-15T21:34:54.884 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553266+0000", 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0636000000000002e-05 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:51 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.26666940199999, 2024-09-15T21:34:54.885 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010200027, 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 51 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.886 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543155+0000", 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543156+0000", 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4199999999999995e-07 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.887 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543163+0000", 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.6839999999999999e-06 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543171+0000", 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5569999999999996e-06 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543179+0000", 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6660000000000008e-06 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.888 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549407+0000", 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0062278159999999997 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549454+0000", 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7509999999999997e-05 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.549536+0000", 2024-09-15T21:34:54.889 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1990999999999995e-05 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552560+0000", 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0030232929999999998 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553303+0000", 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00074331199999999999 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.890 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553332+0000", 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9255e-05 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553355+0000", 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3201e-05 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.891 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:52 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.26662277299999, 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.010277738, 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 52 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.892 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543202+0000", 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.893 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543203+0000", 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543207+0000", 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7659999999999998e-06 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543215+0000", 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2139999999999999e-06 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.894 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.543220+0000", 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.9000000000000003e-06 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550272+0000", 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0070512989999999996 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550352+0000", 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.9887000000000003e-05 2024-09-15T21:34:54.895 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.550470+0000", 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000117859 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.552563+0000", 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020931140000000001 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.896 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553393+0000", 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083056899999999997 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553445+0000", 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.1406000000000002e-05 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:26.553480+0000", 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5160999999999998e-05 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.897 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:68 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.80927892, 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063487919999999998, 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.898 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 68 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000546+0000", 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.899 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000547+0000", 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4300000000000003e-07 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000551+0000", 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6720000000000003e-06 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000560+0000", 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1779999999999992e-06 2024-09-15T21:34:54.900 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000568+0000", 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2409999999999998e-06 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002112+0000", 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0015442120000000001 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.901 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002157+0000", 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4957999999999999e-05 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002233+0000", 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.6599000000000003e-05 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005386+0000", 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0031530629999999998 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.902 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006824+0000", 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014377859999999999 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006858+0000", 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3547000000000003e-05 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006895+0000", 2024-09-15T21:34:54.903 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6893000000000001e-05 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:69 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.80920972999999, 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063469429999999999, 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.904 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 69 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000615+0000", 2024-09-15T21:34:54.905 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000616+0000", 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1299999999999998e-07 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000790+0000", 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00017395 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.906 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000801+0000", 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1185e-05 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000807+0000", 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0889999999999996e-06 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002454+0000", 2024-09-15T21:34:54.907 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001646319 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002502+0000", 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8881999999999997e-05 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.002584+0000", 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.1619999999999994e-05 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.908 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005419+0000", 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.002834844 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006902+0000", 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0014834990000000001 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006937+0000", 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.4944000000000003e-05 2024-09-15T21:34:54.909 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006962+0000", 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4698e-05 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:72 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.80894447399999, 2024-09-15T21:34:54.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064138119999999996, 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 72 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:54.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000880+0000", 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000881+0000", 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1899999999999998e-07 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000914+0000", 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.3501000000000003e-05 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000925+0000", 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0474e-05 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000943+0000", 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.8117e-05 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003630+0000", 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0026874609999999999 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003677+0000", 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7181000000000001e-05 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.003749+0000", 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1783e-05 2024-09-15T21:34:54.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005500+0000", 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001750419 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007210+0000", 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001710208 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007254+0000", 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.4122999999999998e-05 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007294+0000", 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.0225999999999997e-05 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:73 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.916 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.80887020399999, 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064174649999999998, 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 73 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000955+0000", 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.6699999999999999e-07 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000963+0000", 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.5390000000000001e-06 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000967+0000", 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.25e-06 2024-09-15T21:34:54.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004405+0000", 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0034378030000000001 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004454+0000", 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.8702999999999998e-05 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004528+0000", 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4448000000000003e-05 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006191+0000", 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016622290000000001 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007301+0000", 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0011103879999999999 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007342+0000", 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.1199000000000003e-05 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007372+0000", 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9833999999999999e-05 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.922 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:74 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.80884130300001, 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0064677830000000004, 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.923 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 74 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000983+0000", 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000984+0000", 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.27e-07 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000986+0000", 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6599999999999999e-06 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000991+0000", 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.138e-06 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000994+0000", 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.6739999999999999e-06 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004786+0000", 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0037912309999999999 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004833+0000", 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7809999999999998e-05 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.004905+0000", 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.1464000000000006e-05 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006227+0000", 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001322679 2024-09-15T21:34:54.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007377+0000", 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001149611 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007420+0000", 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2787000000000002e-05 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007451+0000", 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1402000000000002e-05 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:75 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.80882597499999, 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0095602970000000006, 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.929 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 75 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.000999+0000", 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5800000000000001e-07 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001063+0000", 2024-09-15T21:34:54.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.4247999999999999e-05 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001070+0000", 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.336e-06 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001074+0000", 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.8879999999999999e-06 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005523+0000", 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0044495450000000001 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005577+0000", 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3593e-05 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.005654+0000", 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7649000000000001e-05 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007457+0000", 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0018022349999999999 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010505+0000", 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003048346 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010535+0000", 2024-09-15T21:34:54.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9802e-05 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010559+0000", 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.4397e-05 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4198.0:76 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-15T21:34:54.935 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.80874168700001, 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0097515940000000006, 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4198", 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/3136721100", 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 76 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:34:54.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001083+0000", 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001084+0000", 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.4199999999999996e-07 2024-09-15T21:34:54.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001087+0000", 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9459999999999998e-06 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001090+0000", 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.569e-06 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.001093+0000", 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.813e-06 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006252+0000", 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0051585290000000002 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006299+0000", 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7144000000000001e-05 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 0", 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.006372+0000", 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3084999999999996e-05 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.007462+0000", 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0010899250000000001 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:34:54.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010778+0000", 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0033161330000000002 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010810+0000", 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.1735e-05 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:27.010835+0000", 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.5173000000000001e-05 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.942 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:34:54.942 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:34:54.942 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:54.943 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:54.944 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:54.941+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:55.044 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:55.043+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:55.057 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:55.057 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:55.057 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:55.057 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:55.066 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:55.145 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:55.143+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:55.174 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:55.174 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:55.174 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:55.174 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:55.174 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:55.183 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:55.245 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:55.244+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:55.331 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:55.331 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:55.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:55.331 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:20 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'10, mlcod=17'8)", 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.88788780600001, 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017142940000000001, 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441134+0000", 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441135+0000", 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7000000000000003e-07 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441261+0000", 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000126821 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441275+0000", 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3407999999999999e-05 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441281+0000", 2024-09-15T21:34:55.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9200000000000001e-06 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441330+0000", 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8965000000000003e-05 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.441344+0000", 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4635e-05 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442813+0000", 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001468757 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442848+0000", 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4918e-05 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:21 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'11, mlcod=17'8)", 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.88697477400001, 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001628937, 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442047+0000", 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442048+0000", 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7700000000000004e-07 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442130+0000", 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2227999999999999e-05 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442146+0000", 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6075000000000001e-05 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442185+0000", 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9529e-05 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442243+0000", 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7846000000000001e-05 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.442260+0000", 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6351e-05 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443648+0000", 2024-09-15T21:34:55.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00138849 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.443676+0000", 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7640999999999998e-05 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:48 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'30, mlcod=17'22)", 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:55.341 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.781106502, 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020461780000000001, 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547915+0000", 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547916+0000", 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.8100000000000002e-07 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547924+0000", 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1829999999999994e-06 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547942+0000", 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8019000000000001e-05 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.547949+0000", 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6880000000000002e-06 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548033+0000", 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.3920999999999998e-05 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548045+0000", 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1963999999999999e-05 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549932+0000", 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001887461 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549961+0000", 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9261e-05 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:49 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'31, mlcod=17'22)", 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.780694232, 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0045952390000000001, 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548327+0000", 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548328+0000", 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.9800000000000003e-07 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548333+0000", 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2650000000000004e-06 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548344+0000", 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0574000000000001e-05 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548351+0000", 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.635e-06 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548401+0000", 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0216000000000002e-05 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548420+0000", 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8804000000000002e-05 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552884+0000", 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044642980000000002 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552923+0000", 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8748999999999998e-05 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:50 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'32, mlcod=17'22)", 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.78024606299999, 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.004176206, 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.351 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:55.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548776+0000", 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.06e-07 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548834+0000", 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8352e-05 2024-09-15T21:34:55.353 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548844+0000", 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.3049999999999998e-06 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548848+0000", 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7040000000000002e-06 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.354 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548888+0000", 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.9542000000000002e-05 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.548901+0000", 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2677000000000001e-05 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.355 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552925+0000", 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0040247449999999997 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552952+0000", 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6274999999999998e-05 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.356 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:51 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'33, mlcod=17'22)", 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.77940771499999, 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0033649859999999999, 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549614+0000", 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7400000000000003e-07 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549618+0000", 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8569999999999997e-06 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549626+0000", 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3989999999999998e-06 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549633+0000", 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3010000000000001e-06 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549666+0000", 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3482999999999998e-05 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.549680+0000", 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3314e-05 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552953+0000", 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032729069999999998 2024-09-15T21:34:55.360 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.552979+0000", 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6151000000000001e-05 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:52 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'34, mlcod=17'28)", 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.77849124900001, 2024-09-15T21:34:55.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0027003280000000001, 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550530+0000", 2024-09-15T21:34:55.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550531+0000", 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4099999999999999e-07 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550534+0000", 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0800000000000002e-06 2024-09-15T21:34:55.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550542+0000", 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0569999999999993e-06 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550547+0000", 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5009999999999998e-06 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.364 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550597+0000", 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0878000000000003e-05 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.550610+0000", 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2665e-05 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553199+0000", 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0025890129999999998 2024-09-15T21:34:55.365 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:26.553231+0000", 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1593000000000001e-05 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:65 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'43, mlcod=17'40)", 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:55.366 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.328063679, 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035718579999999998, 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.367 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000958+0000", 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000959+0000", 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8999999999999996e-07 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000965+0000", 2024-09-15T21:34:55.368 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2559999999999999e-06 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000971+0000", 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6420000000000001e-06 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.000975+0000", 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7129999999999999e-06 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.369 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001132+0000", 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00015674800000000001 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001148+0000", 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5778e-05 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.370 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004503+0000", 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00335549 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004530+0000", 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6641000000000001e-05 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:66 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'44, mlcod=17'40)", 2024-09-15T21:34:55.371 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.32761692, 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0032452409999999998, 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.372 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001405+0000", 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001406+0000", 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001486+0000", 2024-09-15T21:34:55.373 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0257000000000001e-05 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001495+0000", 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0829999999999993e-06 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001502+0000", 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7299999999999999e-06 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.374 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001569+0000", 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7300999999999998e-05 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001593+0000", 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3907e-05 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004611+0000", 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030182199999999998 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.375 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004650+0000", 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8776999999999998e-05 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:67 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'45, mlcod=17'40)", 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.327189678, 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0028489660000000001, 2024-09-15T21:34:55.376 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.377 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001832+0000", 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.0500000000000004e-07 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001836+0000", 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4580000000000002e-06 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.378 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001843+0000", 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7059999999999998e-06 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001848+0000", 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000003e-06 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001887+0000", 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8979000000000001e-05 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.379 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.001899+0000", 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1909999999999999e-05 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004653+0000", 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002754313 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004681+0000", 2024-09-15T21:34:55.380 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7464999999999999e-05 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:68 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'46, mlcod=17'41)", 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.32672272900001, 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024110469999999999, 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.381 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002299+0000", 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.2200000000000004e-07 2024-09-15T21:34:55.382 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002303+0000", 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7529999999999999e-06 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002311+0000", 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.9950000000000005e-06 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.383 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002315+0000", 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.602e-06 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002349+0000", 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4060000000000003e-05 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002361+0000", 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.218e-05 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.384 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004683+0000", 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0023220070000000001 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004710+0000", 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.6828000000000001e-05 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.385 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:69 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'47, mlcod=17'41)", 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.326354186, 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002077387, 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.386 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002667+0000", 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002668+0000", 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6199999999999997e-07 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.387 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002732+0000", 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4082000000000001e-05 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002741+0000", 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.6100000000000006e-06 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002744+0000", 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.67e-06 2024-09-15T21:34:55.388 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002776+0000", 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2019e-05 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.002790+0000", 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3312000000000001e-05 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.389 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004713+0000", 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001922769 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004745+0000", 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2262999999999999e-05 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:70 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'48, mlcod=17'41)", 2024-09-15T21:34:55.390 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.32593711600001, 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001707526, 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.391 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003084+0000", 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003085+0000", 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.7800000000000001e-07 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003089+0000", 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.143e-06 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.392 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003097+0000", 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.8059999999999995e-06 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003101+0000", 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.529e-06 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.393 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003134+0000", 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3043999999999997e-05 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.003148+0000", 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4041e-05 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004748+0000", 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001600261 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.394 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.004792+0000", 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4124e-05 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:75 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'53, mlcod=17'43)", 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.32328609999999, 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0046442530000000001, 2024-09-15T21:34:55.395 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005735+0000", 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.396 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005736+0000", 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1299999999999999e-07 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005803+0000", 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6464999999999995e-05 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005813+0000", 2024-09-15T21:34:55.397 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0414000000000001e-05 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005819+0000", 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.8969999999999996e-06 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005863+0000", 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.4223e-05 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.398 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.005886+0000", 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.3042000000000001e-05 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010347+0000", 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0044605979999999996 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010380+0000", 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.2901000000000002e-05 2024-09-15T21:34:55.399 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:76 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'54, mlcod=17'43)", 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.322588771, 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0042302390000000002, 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.400 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006433+0000", 2024-09-15T21:34:55.401 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6300000000000005e-07 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006438+0000", 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.8609999999999997e-06 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006447+0000", 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4789999999999996e-06 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.402 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.006451+0000", 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8670000000000004e-06 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010009+0000", 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0035584649999999998 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010024+0000", 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5281000000000002e-05 2024-09-15T21:34:55.403 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010635+0000", 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00061075300000000001 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.010663+0000", 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7970000000000002e-05 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.404 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:94 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'68, mlcod=17'61)", 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.039263877, 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0030408459999999998, 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.405 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289758+0000", 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.2e-07 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.406 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289762+0000", 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.732e-06 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289770+0000", 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.1489999999999994e-06 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289774+0000", 2024-09-15T21:34:55.407 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.6049999999999998e-06 2024-09-15T21:34:55.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.408 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289803+0000", 2024-09-15T21:34:55.408 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9524000000000001e-05 2024-09-15T21:34:55.408 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.408 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.408 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.289815+0000", 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1406000000000001e-05 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292764+0000", 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0029493689999999999 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.292799+0000", 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.4541e-05 2024-09-15T21:34:55.409 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:95 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'69, mlcod=17'67)", 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.03527394700001, 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0017143130000000001, 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.410 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293748+0000", 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.3e-07 2024-09-15T21:34:55.411 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293809+0000", 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.1029999999999997e-05 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293818+0000", 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.0920000000000004e-06 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.412 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293823+0000", 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.0509999999999998e-06 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293865+0000", 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2027999999999998e-05 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.293886+0000", 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1761999999999999e-05 2024-09-15T21:34:55.413 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295430+0000", 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0015439550000000001 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.295462+0000", 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1665000000000003e-05 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.414 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:96 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'70, mlcod=17'67)", 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.034617685, 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.003784431, 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:55.415 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294404+0000", 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294405+0000", 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.4499999999999997e-07 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.416 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294481+0000", 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.6806999999999998e-05 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294491+0000", 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.7780000000000002e-06 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294495+0000", 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3409999999999996e-06 2024-09-15T21:34:55.417 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294538+0000", 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.2923000000000002e-05 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.294551+0000", 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2488e-05 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298158+0000", 2024-09-15T21:34:55.418 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0036067320000000001 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298188+0000", 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0716999999999997e-05 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:97 1.0 e17/16 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'71, mlcod=17'69)", 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:55.419 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.03016412700001, 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002154039, 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298857+0000", 2024-09-15T21:34:55.420 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298858+0000", 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.0900000000000001e-07 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298862+0000", 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.3089999999999997e-06 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.421 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298879+0000", 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6201999999999998e-05 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298883+0000", 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6199999999999998e-06 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298938+0000", 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.4666999999999997e-05 2024-09-15T21:34:55.422 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.298951+0000", 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.3387000000000001e-05 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.300983+0000", 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0020317880000000001 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.423 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301012+0000", 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8456999999999999e-05 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4198.0:98 1.0 e17/16 1:30306672:devicehealth::main.db.0000000000000000:head v 17'72, mlcod=17'70)", 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.027431372, 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0022031220000000001, 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.424 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.425 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.425 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.425 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.429 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301590+0000", 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301591+0000", 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.3099999999999997e-07 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.430 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301595+0000", 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5279999999999997e-06 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301605+0000", 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.4890000000000001e-06 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301609+0000", 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.561e-06 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.431 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301643+0000", 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.3793e-05 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.301654+0000", 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0701e-05 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.432 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303768+0000", 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.002113752 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:27.303793+0000", 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.5667e-05 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.433 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:55.435 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:55.435 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:55.433+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:55.536 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:55.534+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:55.583 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:55.583 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:55.583 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:55.583 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:55.592 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:55.676 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:55.674+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:55.737 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:55.735+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:55.739 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:55.739 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:55.739 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:55.739 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:55.739 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:55.748 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:55.837 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:55.836+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:55.896 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:55.896 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:55.896 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:55.896 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:55.896 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 203.43745201900001, 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.897 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.898 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.899 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:55.900 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 202.43157062700001, 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:55.901 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.902 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:55.903 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.904 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:55.905 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 62.913051230999997, 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.906 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.907 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:55.908 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:55.909 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:55.910 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:55.910 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:55.910 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.910 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.910 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.910 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:55.910 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:55.910 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:55.911 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:55.922 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:55.938 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:55.936+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:56.039 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:56.037+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:56.062 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:56.062 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:56.063 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:56.063 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:56.072 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:56.098 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-15T21:34:56.140 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:56.138+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:56.222 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:56.222 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:56.222 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:56.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:56.241 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:56.239+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:56.331 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":62,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":36,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:56.340 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:56.340 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:56.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:56.340 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 204.88291645699999, 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:56.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:56.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:56.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:56.345 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 119.22052606, 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:56.346 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.347 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:56.348 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 104.850492151, 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:56.349 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.350 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:56.351 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.352 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.354 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:56.354 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:56.354 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:56.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.354 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:56.354 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.354 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 99.122004658999998, 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:56.355 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:56.356 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:56.356 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.356 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:56.356 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.356 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:56.356 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:56.356 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.357 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:56.358 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.359 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 39.120019423000002, 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001278429, 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:56.360 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 5 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.361 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219037+0000", 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1600000000000001e-06 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219049+0000", 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2153999999999999e-05 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219070+0000", 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0441e-05 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.362 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219090+0000", 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0665e-05 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220231+0000", 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001140655 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220313+0000", 2024-09-15T21:34:56.363 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2354000000000005e-05 2024-09-15T21:34:56.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.364 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:56.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.364 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:56.364 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:56.364 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:56.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:56.366 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:56.364+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:56.420 INFO:teuthology.orchestra.run.smithi019.stderr:2024-09-15T21:34:56.418+0000 7fb9a530f640 0 --2- 172.21.15.19:0/649443288 >> v2:172.21.15.19:3300/0 conn(0x7fb9a00b64d0 0x7fb9a0146b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-15T21:34:56.466 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:56.464+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:56.475 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:56.475 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:56.475 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:56.476 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:56.484 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:56.549 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:34:56.569 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:56.567+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:56.633 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:56.633 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:56.634 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:56.634 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:56.634 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:56.643 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:56.689 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:56.687+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:56.697 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:56.697 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:56.716 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:56.716 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:56.732 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:56.733 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:56.749 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:56.750 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:56.755 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:56.755 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:56.764 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:34:56.768 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:56.768 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:56.769 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:56.766+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:56.785 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:56.786 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:56.803 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:56.803 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:56.821 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:56.821 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:56.868 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:56.866+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:56.882 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:56.882 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:56.891 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:34:56.975 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:56.973+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:57.036 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:57.036 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:57.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:34:57.069 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:57.068+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:57.095 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is set 2024-09-15T21:34:57.170 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:57.168+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:57.189 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:57.189 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:57.198 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:34:57.282 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:57.280+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:57.341 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:57.342 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:57.350 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:34:57.372 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:57.370+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:57.472 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:57.470+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:57.493 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:57.493 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:57.501 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:34:57.585 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:57.583+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:57.644 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:57.644 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:57.653 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:34:57.694 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:57.692+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:57.743 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:34:57.743 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:34:57.743 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-15T21:34:57.743 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-15T21:34:57.743 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-15T21:34:57.744 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok config set filestore_inject_stall 3 2024-09-15T21:34:57.788 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:34:57.786+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:34:57.800 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:57.800 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:57.809 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:34:57.849 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:34:57.849 INFO:teuthology.orchestra.run.smithi192.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-15T21:34:57.849 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:34:57.875 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:57.874+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:57.951 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:57.951 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:57.959 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:34:58.003 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:58.001+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:58.062 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:58.062 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:58.071 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:34:58.113 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:58.110+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:58.177 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:58.175+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:58.215 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:58.215 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:58.225 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:34:58.278 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:58.276+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:58.367 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:58.368 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:58.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:34:58.379 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:58.377+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:58.479 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:58.479 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:58.479 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:58.478+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:58.487 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:34:58.580 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:58.578+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:58.607 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:58.608 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:58.617 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:34:58.701 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:58.699+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:58.767 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:58.767 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:58.775 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:34:58.817 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:34:58.815+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:34:58.833 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-15T21:34:58.903 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:58.901+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:58.925 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:34:58.925 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:34:58.938 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:34:58.984 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:34:58.984 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:34:58.985 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:34:58.983+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:34:59.003 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:34:59.003 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:34:59.019 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:34:59.019 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:34:59.036 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:34:59.036 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:34:59.053 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:34:59.053 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:34:59.070 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:34:59.070 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:34:59.083 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:59.082+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:34:59.087 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:34:59.087 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:34:59.088 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:59.088 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:59.088 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:59.088 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:59.097 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:34:59.104 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:34:59.105 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:34:59.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:34:59.185 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:59.183+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:59.212 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:59.212 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:59.212 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:59.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:34:59.286 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:59.285+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:5 2.2 e21/19 2:5cea7035:::rbd_info:head v 21'1, mlcod=0'0)", 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 206.87198244000001, 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00088376800000000005, 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:59.331 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.332 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457618+0000", 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.11e-07 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457628+0000", 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0217999999999999e-05 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457645+0000", 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7227e-05 2024-09-15T21:34:59.333 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457660+0000", 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4508000000000001e-05 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457752+0000", 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 9.1629e-05 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:59.334 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.457772+0000", 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0670999999999999e-05 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458454+0000", 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00068178599999999998 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:32.458502+0000", 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.7518000000000001e-05 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.335 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:7 2.2 e22/19 2:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 205.86610104799999, 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000971753, 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.336 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463499+0000", 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8e-07 2024-09-15T21:34:59.337 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463505+0000", 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.9209999999999997e-06 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463522+0000", 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.7167e-05 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463533+0000", 2024-09-15T21:34:59.338 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0224000000000001e-05 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463610+0000", 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.7330999999999995e-05 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.463639+0000", 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.8968000000000001e-05 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.339 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464424+0000", 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00078543899999999997 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:33.464471+0000", 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.6523000000000002e-05 2024-09-15T21:34:59.340 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.2 53/52 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 66.347581652000002, 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00031055600000000003, 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.341 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982019+0000", 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.8299999999999998e-07 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.342 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982037+0000", 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.8074999999999999e-05 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982052+0000", 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5260999999999998e-05 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982063+0000", 2024-09-15T21:34:59.343 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1173e-05 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982125+0000", 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.19e-05 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:59.344 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982134+0000", 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.7469999999999994e-06 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:52.982329+0000", 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00019501699999999999 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.345 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:59.349 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:34:59.366 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:34:59.387 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:59.385+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:59.458 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:59.458 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:59.458 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:59.459 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:59.468 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:34:59.488 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:59.486+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:59.578 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:59.578 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:59.578 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:59.578 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:59.578 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:59.587 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:34:59.588 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:34:59.586+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:34:59.689 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:34:59.688+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:34:59.694 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:59.694 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:34:59.694 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:34:59.694 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:34:59.694 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.694 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_repop(client.4238.0:1 2.3 e20/19 2:c4c92e5a:::rbd_trash:head v 20'1, mlcod=0'0)", 2024-09-15T21:34:59.694 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 208.23666983999999, 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0026613980000000001, 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:59.695 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456138+0000", 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1199999999999999e-07 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.696 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456145+0000", 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.1219999999999997e-06 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456160+0000", 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4358e-05 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456172+0000", 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.1851000000000001e-05 2024-09-15T21:34:59.697 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456238+0000", 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.6440999999999996e-05 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.456267+0000", 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.9006999999999999e-05 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "commit_sent", 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458750+0000", 2024-09-15T21:34:59.698 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.0024830220000000001 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:31:31.458799+0000", 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.9385000000000002e-05 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPushReply(2.3 37/36 [PushReplyOp(2:c4c92e5a:::rbd_trash:head)])", 2024-09-15T21:34:59.699 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 122.57427944299999, 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00017548600000000001, 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.700 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118528+0000", 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118529+0000", 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.0699999999999998e-07 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118531+0000", 2024-09-15T21:34:59.701 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.7039999999999999e-06 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118536+0000", 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.5970000000000002e-06 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118541+0000", 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.3070000000000002e-06 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.702 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118579+0000", 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.7849999999999998e-05 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118704+0000", 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000124721 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.703 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPull(2.3 e41/40 cost 1049576)", 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 108.20424553399999, 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016133799999999999, 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:59.704 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488562+0000", 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488563+0000", 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 4.63e-07 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.705 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488575+0000", 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2619000000000001e-05 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488583+0000", 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.0790000000000001e-06 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:59.706 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488590+0000", 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 6.7240000000000003e-06 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488621+0000", 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 3.1025999999999997e-05 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:11.488724+0000", 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000102427 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.707 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:3 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e26)", 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 102.475758042, 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00070331899999999999, 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:59.708 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 3 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217050+0000", 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.709 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217052+0000", 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.9300000000000002e-06 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217062+0000", 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0729e-05 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217079+0000", 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.6311999999999999e-05 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.710 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217088+0000", 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.8689999999999993e-06 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217669+0000", 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00058094600000000004 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:33:17.217753+0000", 2024-09-15T21:34:59.711 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.4532999999999999e-05 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "osd_op(client.4204.0:5 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 42.473772805999999, 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001278429, 2024-09-15T21:34:59.712 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "reached pg", 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "client_info": { 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "client": "client.4204", 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "tid": 5 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.713 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219035+0000", 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219037+0000", 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.1600000000000001e-06 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:34:59.714 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219049+0000", 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.2153999999999999e-05 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219070+0000", 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0441e-05 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.219090+0000", 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 2.0665e-05 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.715 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220231+0000", 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.001140655 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:34:17.220313+0000", 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 8.2354000000000005e-05 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:34:59.716 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:59.717 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:34:59.769 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":62,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":39,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:34:59.791 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:34:59.789+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:34:59.826 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:59.827 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:59.827 INFO:teuthology.orchestra.run.smithi019.stdout: "num_ops": 0 2024-09-15T21:34:59.827 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:59.835 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:34:59.891 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:34:59.890+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:34:59.941 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:34:59.941 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [], 2024-09-15T21:34:59.942 INFO:teuthology.orchestra.run.smithi019.stdout: "complaint_time": 30, 2024-09-15T21:34:59.942 INFO:teuthology.orchestra.run.smithi019.stdout: "num_blocked_ops": 0 2024-09-15T21:34:59.942 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:34:59.950 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:34:59.993 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:34:59.992+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:35:00.057 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 20, 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 600, 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "ops": [ 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "description": "MOSDPGPush(2.3 37/36 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "initiated_at": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "age": 122.93855690300001, 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.000288619, 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "continuous": false, 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "type_data": { 2024-09-15T21:35:00.058 INFO:teuthology.orchestra.run.smithi019.stdout: "flag_point": "started", 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "events": [ 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "initiated", 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "header_read", 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117903+0000", 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "throttled", 2024-09-15T21:35:00.059 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117904+0000", 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 5.6700000000000003e-07 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "all_read", 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117918+0000", 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4182000000000001e-05 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "dispatched", 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117933+0000", 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.5092000000000001e-05 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.060 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.117944+0000", 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.0532e-05 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "reached_pg", 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118015+0000", 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 7.0964999999999996e-05 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "started", 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118029+0000", 2024-09-15T21:35:00.061 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 1.4623e-05 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: "event": "done", 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: "time": "2024-09-15T21:32:57.118192+0000", 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: "duration": 0.00016265799999999999 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:00.062 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:00.066 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:35:00.104 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:35:00.111 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:35:00.109+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:35:00.184 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:00.184 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:35:00.184 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:35:00.184 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:00.194 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:35:00.235 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:35:00.233+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:35:00.296 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:35:00.294+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:35:00.338 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:00.338 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:35:00.338 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:35:00.338 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:35:00.338 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:00.346 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:35:00.431 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:35:00.429+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [] 2024-09-15T21:35:00.491 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:00.497 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:35:00.496+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:35:00.500 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:35:00.599 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:35:00.597+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:35:00.645 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:00.654 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:35:00.700 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:35:00.698+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:35:00.796 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:00.796 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:35:00.796 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:35:00.796 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:35:00.796 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:00.801 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:35:00.799+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:35:00.804 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:35:00.901 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:35:00.899+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:1 2.3 2:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 209.45444402699999, 2024-09-15T21:35:00.909 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0063473490000000004, 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 1 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:35:00.910 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452790+0000", 2024-09-15T21:35:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.911 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452792+0000", 2024-09-15T21:35:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3329999999999999e-06 2024-09-15T21:35:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.912 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452803+0000", 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1052e-05 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452820+0000", 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6597000000000001e-05 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.452835+0000", 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4758e-05 2024-09-15T21:35:00.913 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453079+0000", 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000244236 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for peered", 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.453088+0000", 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.3889999999999994e-06 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.914 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455680+0000", 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0025914039999999998 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.455861+0000", 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000181056 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 2", 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.456008+0000", 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014747999999999999 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.915 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.457394+0000", 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013851810000000001 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459030+0000", 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016364730000000001 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459093+0000", 2024-09-15T21:35:00.916 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3448999999999995e-05 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459137+0000", 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3940999999999998e-05 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:1 2.3 e20/19 ondisk, result = 0)", 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:35:00.917 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 209.44830439699999, 2024-09-15T21:35:00.918 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00021493599999999999, 2024-09-15T21:35:00.918 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.918 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.918 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:35:00.918 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.918 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.918 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.918 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:35:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:35:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.919 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458930+0000", 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3e-07 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458936+0000", 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.2089999999999996e-06 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.920 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458947+0000", 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.113e-05 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.458955+0000", 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.2630000000000003e-06 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.921 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459016+0000", 2024-09-15T21:35:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0983999999999998e-05 2024-09-15T21:35:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459025+0000", 2024-09-15T21:35:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5390000000000007e-06 2024-09-15T21:35:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.922 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.459145+0000", 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000120081 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.923 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:2 2.2 2:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 209.44513622100001, 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00025157499999999998, 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 2 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.924 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462098+0000", 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.925 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462100+0000", 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.2369999999999999e-06 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462113+0000", 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2622999999999999e-05 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.926 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462142+0000", 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.9615999999999999e-05 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462157+0000", 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4453e-05 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.927 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462258+0000", 2024-09-15T21:35:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000101038 2024-09-15T21:35:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462349+0000", 2024-09-15T21:35:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1607999999999994e-05 2024-09-15T21:35:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.928 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:3 2.2 2:5cea7035:::rbd_info:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e20)", 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 209.444590468, 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00015530399999999999, 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.929 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 3 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.930 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462644+0000", 2024-09-15T21:35:00.931 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.75e-07 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462648+0000", 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2899999999999998e-06 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462656+0000", 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.8999999999999995e-06 2024-09-15T21:35:00.932 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462666+0000", 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4660000000000005e-06 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462714+0000", 2024-09-15T21:35:00.933 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.7682999999999997e-05 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:31.462799+0000", 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5389999999999994e-05 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:5 2.2 2:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:35:00.934 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.45195215999999, 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.003434257, 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 5 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.935 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455282+0000", 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455284+0000", 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.063e-06 2024-09-15T21:35:00.936 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455294+0000", 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.027e-05 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455315+0000", 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0248999999999999e-05 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.937 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455327+0000", 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2835e-05 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455400+0000", 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.2471999999999993e-05 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.455406+0000", 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.6629999999999998e-06 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.938 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457128+0000", 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0017220650000000001 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457233+0000", 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000105376 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.457352+0000", 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00011869199999999999 2024-09-15T21:35:00.939 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458184+0000", 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00083233199999999997 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458655+0000", 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00047073200000000002 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:35:00.940 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458687+0000", 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.2277999999999998e-05 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458716+0000", 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.923e-05 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:5 2.2 e20/19 ondisk, result = 0)", 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:35:00.941 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 208.44865933200001, 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00014930799999999999, 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.942 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458575+0000", 2024-09-15T21:35:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458576+0000", 2024-09-15T21:35:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.043e-06 2024-09-15T21:35:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.943 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458581+0000", 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.079e-06 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458594+0000", 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.3162999999999999e-05 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.944 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458602+0000", 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.7880000000000007e-06 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458645+0000", 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3037999999999997e-05 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458650+0000", 2024-09-15T21:35:00.945 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.3979999999999998e-06 2024-09-15T21:35:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:32.458724+0000", 2024-09-15T21:35:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.3799000000000003e-05 2024-09-15T21:35:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.946 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4238.0:7 2.2 2:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.44610856200001, 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0035964349999999998, 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:35:00.947 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4238", 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.19:0/2265702413", 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 7 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.948 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461126+0000", 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461128+0000", 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.475e-06 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461138+0000", 2024-09-15T21:35:00.949 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.696e-06 2024-09-15T21:35:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461157+0000", 2024-09-15T21:35:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.9655000000000001e-05 2024-09-15T21:35:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.950 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461167+0000", 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7589999999999992e-06 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461245+0000", 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8263000000000006e-05 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op must wait for map", 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.461255+0000", 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2240000000000001e-06 2024-09-15T21:35:00.951 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.462976+0000", 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001721841 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463101+0000", 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012413600000000001 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.952 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "waiting for subops from 1", 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.463287+0000", 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000185949 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "op_commit", 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464061+0000", 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00077425900000000001 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.953 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "sub_op_commit_rec", 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464616+0000", 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00055509799999999996 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "commit_sent", 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464679+0000", 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3039000000000004e-05 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464722+0000", 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.3040999999999997e-05 2024-09-15T21:35:00.954 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_repop_reply(client.4238.0:7 2.2 e21/19 ondisk, result = 0)", 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 207.44270787299999, 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00020529000000000001, 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.955 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464526+0000", 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.956 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464527+0000", 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4600000000000003e-07 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464531+0000", 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.2729999999999996e-06 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464543+0000", 2024-09-15T21:35:00.957 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1589e-05 2024-09-15T21:35:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464549+0000", 2024-09-15T21:35:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.331e-06 2024-09-15T21:35:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.958 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464599+0000", 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.0102999999999998e-05 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464610+0000", 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0496000000000001e-05 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.959 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:31:33.464732+0000", 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012165199999999999 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.960 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.3 41/40 [PushOp(2:c4c92e5a:::rbd_trash:head, version: 20'1, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:c4c92e5a:::rbd_trash:head@20'1, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.41847273499999, 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000347969, 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.961 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488761+0000", 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488762+0000", 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 4.39e-07 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488771+0000", 2024-09-15T21:35:00.962 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.4809999999999995e-06 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488783+0000", 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1286e-05 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488793+0000", 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0458e-05 2024-09-15T21:35:00.963 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488855+0000", 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2417999999999998e-05 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.488864+0000", 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5750000000000001e-06 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.964 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.489109+0000", 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00024531200000000003 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 41/40 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.41546657000001, 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001147035, 2024-09-15T21:35:00.965 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.966 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491768+0000", 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4600000000000004e-07 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491781+0000", 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2863e-05 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.967 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491802+0000", 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.0905999999999999e-05 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491819+0000", 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6765e-05 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491889+0000", 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.0414999999999996e-05 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.968 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.491899+0000", 2024-09-15T21:35:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.7899999999999994e-06 2024-09-15T21:35:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.492915+0000", 2024-09-15T21:35:00.969 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00101555 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 37.992492116999998, 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00023751800000000001, 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:00.970 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 6 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.971 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914742+0000", 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914744+0000", 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5999999999999999e-06 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:00.972 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914750+0000", 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.3389999999999998e-06 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914759+0000", 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.2e-06 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914768+0000", 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.4689999999999997e-06 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.973 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914858+0000", 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.0499000000000002e-05 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:34:22.914980+0000", 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00012141099999999999 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.974 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:00.975 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:00.975 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:00.976 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:35:01.002 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:35:01.001+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:35:01.079 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:01.079 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:35:01.079 INFO:teuthology.orchestra.run.smithi192.stdout: "num_ops": 0 2024-09-15T21:35:01.079 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:01.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:35:01.103 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:35:01.101+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:35:01.119 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-15T21:35:01.191 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:01.191 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [], 2024-09-15T21:35:01.191 INFO:teuthology.orchestra.run.smithi192.stdout: "complaint_time": 30, 2024-09-15T21:35:01.191 INFO:teuthology.orchestra.run.smithi192.stdout: "num_blocked_ops": 0 2024-09-15T21:35:01.191 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:01.200 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:35:01.203 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:35:01.202+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:35:01.260 INFO:teuthology.orchestra.run.smithi019.stdout:osd.0: {} 2024-09-15T21:35:01.260 INFO:teuthology.orchestra.run.smithi019.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-15T21:35:01.277 INFO:teuthology.orchestra.run.smithi019.stdout:osd.1: {} 2024-09-15T21:35:01.277 INFO:teuthology.orchestra.run.smithi019.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-15T21:35:01.293 INFO:teuthology.orchestra.run.smithi019.stdout:osd.2: {} 2024-09-15T21:35:01.293 INFO:teuthology.orchestra.run.smithi019.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-15T21:35:01.302 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2024-09-15T21:35:01.302 INFO:teuthology.orchestra.run.smithi192.stdout: "size": 20, 2024-09-15T21:35:01.302 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 600, 2024-09-15T21:35:01.302 INFO:teuthology.orchestra.run.smithi192.stdout: "ops": [ 2024-09-15T21:35:01.302 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(2.2 26/25 [PushOp(2:5cea7035:::rbd_info:head, version: 22'2, data_included: [0~19], data_size: 19, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(2:5cea7035:::rbd_info:head@22'2, size: 19, copy_subset: [(0, 19)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 19, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 158.38806227699999, 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00056538599999999997, 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.303 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913107+0000", 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8999999999999996e-07 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.304 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913135+0000", 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7302000000000001e-05 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913151+0000", 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.6177000000000001e-05 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913163+0000", 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.2480000000000001e-05 2024-09-15T21:35:01.305 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913224+0000", 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.0439999999999997e-05 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913236+0000", 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.1751e-05 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.306 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:22.913672+0000", 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000436546 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'71, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'71, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 150.361724858, 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.001722732, 2024-09-15T21:35:01.307 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939444+0000", 2024-09-15T21:35:01.308 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939446+0000", 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.705e-06 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939715+0000", 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000269635 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.309 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939743+0000", 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.7276000000000001e-05 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939766+0000", 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.3621e-05 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939835+0000", 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.8891000000000002e-05 2024-09-15T21:35:01.310 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.939845+0000", 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.815e-06 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.941167+0000", 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0013217890000000001 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.311 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPush(1.0 28/27 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 17'72, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@17'72, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 150.35829158999999, 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0020713760000000002, 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "started", 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:35:01.312 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942877+0000", 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.942878+0000", 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.3799999999999996e-07 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:01.313 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943188+0000", 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031010800000000002 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943204+0000", 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.5851000000000001e-05 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943215+0000", 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.0594000000000001e-05 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.314 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943277+0000", 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.2162000000000006e-05 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "started", 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.943286+0000", 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 9.1940000000000004e-06 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:32:30.944949+0000", 2024-09-15T21:35:01.315 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.0016626290000000001 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "MOSDPGPushReply(2.2 41/40 [PushReplyOp(2:5cea7035:::rbd_info:head)])", 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 109.80769001, 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00031497799999999999, 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:35:01.316 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493479+0000", 2024-09-15T21:35:01.317 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 2.6300000000000001e-07 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493483+0000", 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 3.5410000000000001e-06 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493491+0000", 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.8320000000000006e-06 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.318 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493498+0000", 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 7.4789999999999999e-06 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493565+0000", 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 6.7051999999999994e-05 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:11.493794+0000", 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.00022881099999999999 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.319 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "description": "osd_op(client.4204.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "initiated_at": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "age": 98.388107386000001, 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0.000175623, 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "continuous": false, 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "type_data": { 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "flag_point": "reached pg", 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_info": { 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client": "client.4204", 2024-09-15T21:35:01.320 INFO:teuthology.orchestra.run.smithi192.stdout: "client_addr": "172.21.15.192:0/1031645565", 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "tid": 4 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "events": [ 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "initiated", 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "header_read", 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913062+0000", 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 0 2024-09-15T21:35:01.321 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "throttled", 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913063+0000", 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.516e-06 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "all_read", 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913071+0000", 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0209999999999999e-06 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "dispatched", 2024-09-15T21:35:01.322 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913086+0000", 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 1.4543e-05 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "queued_for_pg", 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913094+0000", 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.0770000000000008e-06 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "reached_pg", 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913152+0000", 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 5.8322000000000002e-05 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2024-09-15T21:35:01.323 INFO:teuthology.orchestra.run.smithi192.stdout: { 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "event": "done", 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "time": "2024-09-15T21:33:22.913237+0000", 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: "duration": 8.5143999999999997e-05 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: } 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout: ] 2024-09-15T21:35:01.324 INFO:teuthology.orchestra.run.smithi192.stdout:} 2024-09-15T21:35:01.325 INFO:teuthology.orchestra.run.smithi019.stdout:osd.3: {} 2024-09-15T21:35:01.325 INFO:teuthology.orchestra.run.smithi019.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-15T21:35:01.326 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:35:01.324+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:35:01.326 INFO:teuthology.orchestra.run.smithi019.stdout:osd.4: {} 2024-09-15T21:35:01.327 INFO:teuthology.orchestra.run.smithi019.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-15T21:35:01.327 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:35:01.343 INFO:teuthology.orchestra.run.smithi019.stdout:osd.5: {} 2024-09-15T21:35:01.343 INFO:teuthology.orchestra.run.smithi019.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-15T21:35:01.361 INFO:teuthology.orchestra.run.smithi019.stdout:osd.6: {} 2024-09-15T21:35:01.361 INFO:teuthology.orchestra.run.smithi019.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-15T21:35:01.378 INFO:teuthology.orchestra.run.smithi019.stdout:osd.7: {} 2024-09-15T21:35:01.378 INFO:teuthology.orchestra.run.smithi019.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-15T21:35:01.431 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:35:01.429+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:35:01.431 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:01.431 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:01.440 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:35:01.528 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:35:01.526+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:35:01.543 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:01.543 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:01.551 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:35:01.636 INFO:tasks.ceph.osd.6.smithi192.stderr:2024-09-15T21:35:01.634+0000 7efd5c9d7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33321) UID: 0 2024-09-15T21:35:01.696 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:01.696 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:01.705 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:35:01.746 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:35:01.744+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:35:01.815 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:01.815 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:01.824 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-15T21:35:01.830 INFO:tasks.ceph.osd.7.smithi192.stderr:2024-09-15T21:35:01.829+0000 7f3ad29a7640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33326) UID: 0 2024-09-15T21:35:01.931 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:01.931 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:01.931 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:35:01.929+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:35:01.940 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-15T21:35:02.032 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:35:02.030+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:35:02.087 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:02.088 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:02.096 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:35:02.119 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:35:02.137 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:35:02.135+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:35:02.233 INFO:tasks.ceph.osd.1.smithi019.stderr:2024-09-15T21:35:02.231+0000 7fbee32b4640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33642) UID: 0 2024-09-15T21:35:02.246 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:02.246 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:02.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-15T21:35:02.341 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:35:02.339+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:35:02.404 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:02.404 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:02.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-15T21:35:02.454 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:35:02.452+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:35:02.535 INFO:tasks.ceph.osd.5.smithi192.stderr:2024-09-15T21:35:02.534+0000 7f0542473640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33322) UID: 0 2024-09-15T21:35:02.560 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:02.560 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:02.568 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:35:02.636 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:35:02.634+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:35:02.676 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:02.677 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:02.685 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-15T21:35:02.725 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:35:02.740 INFO:tasks.ceph.osd.2.smithi019.stderr:2024-09-15T21:35:02.738+0000 7f9b93849640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33652) UID: 0 2024-09-15T21:35:02.784 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-15T21:35:02.834 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:02.834 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:02.837 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:35:02.835+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:35:02.844 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-15T21:35:02.859 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 6, 5, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-15T21:35:02.859 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-15T21:35:02.859 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-15T21:35:02.860 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-15T21:35:02.938 INFO:tasks.ceph.osd.0.smithi019.stderr:2024-09-15T21:35:02.936+0000 7f1cc58bd640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33646) UID: 0 2024-09-15T21:35:02.954 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:02.954 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:02.964 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-15T21:35:03.039 INFO:tasks.ceph.osd.4.smithi192.stderr:2024-09-15T21:35:03.037+0000 7f2658987640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 33317) UID: 0 2024-09-15T21:35:03.114 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:03.114 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:03.123 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-15T21:35:03.167 INFO:tasks.ceph.osd.3.smithi019.stderr:2024-09-15T21:35:03.165+0000 7f737ba27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33657) UID: 0 2024-09-15T21:35:03.168 INFO:teuthology.orchestra.run.smithi019.stdout:{"election_epoch":64,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","modified":"2024-09-15T21:30:32.924046Z","created":"2024-09-15T21:30:32.924046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3300","nonce":0},{"type":"v1","addr":"172.21.15.19:6789","nonce":0}]},"addr":"172.21.15.19:6789/0","public_addr":"172.21.15.19:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:3300","nonce":0},{"type":"v1","addr":"172.21.15.192:6789","nonce":0}]},"addr":"172.21.15.192:6789/0","public_addr":"172.21.15.192:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:3301","nonce":0},{"type":"v1","addr":"172.21.15.19:6790","nonce":0}]},"addr":"172.21.15.19:6790/0","public_addr":"172.21.15.19:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-15T21:35:03.181 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-09-15T21:35:03.182 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-15T21:35:03.191 INFO:tasks.thrashosds:joining thrashosds 2024-09-15T21:35:03.195 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: "epoch": 80, 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: "fsid": "13019d66-d4b6-4e2a-8322-38bcb4210bcf", 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: "created": "2024-09-15T21:31:12.329884+0000", 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: "modified": "2024-09-15T21:35:02.722933+0000", 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_up_change": "2024-09-15T21:31:25.414443+0000", 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: "last_in_change": "2024-09-15T21:33:28.789828+0000", 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-15T21:35:03.196 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_num": 5799936, 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_set": [ 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "pglog_hardlimit", 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "purged_snapdirs", 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "recovery_deletes", 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "sortbitwise" 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_version": 5, 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "full_ratio": 0.94999998807907104, 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-15T21:35:03.197 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_snapshot": "", 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_max": 2, 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "max_osd": 8, 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "require_min_compat_client": "luminous", 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "min_compat_client": "luminous", 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "require_osd_release": "squid", 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "allow_crimson": false, 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "pools": [ 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 1, 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": ".mgr", 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:22.622713+0000", 2024-09-15T21:35:03.198 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 1, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool", 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:35:03.199 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 1, 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 1, 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 1, 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 1, 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 1, 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:35:03.200 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "18", 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 0, 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 0, 2024-09-15T21:35:03.201 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:35:03.202 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:35:03.203 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "options": { 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_max": 32, 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_min": 1 2024-09-15T21:35:03.204 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "mgr": {} 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 6.9000000953674316, 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 6.9000000953674316, 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 0.28999999165534973, 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 2, 2024-09-15T21:35:03.205 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 2, 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 0.85879135131835938, 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "pool": 2, 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_name": "rbd", 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "create_time": "2024-09-15T21:31:30.313202+0000", 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags": 8193, 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-15T21:35:03.206 INFO:teuthology.orchestra.run.smithi019.stdout: "type": 1, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "size": 2, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "min_size": 1, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_rule": 0, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_count": 0, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_target": 0, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_barrier": 0, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "is_stretch_pool": false, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "object_hash": 2, 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_autoscale_mode": "off", 2024-09-15T21:35:03.207 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num": 8, 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num": 8, 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_placement_num_target": 8, 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_target": 8, 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_num_pending": 8, 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "last_pg_merge_meta": { 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "source_pgid": "0.0", 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "ready_epoch": 0, 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_started": 0, 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "last_epoch_clean": 0, 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "source_version": "0'0", 2024-09-15T21:35:03.208 INFO:teuthology.orchestra.run.smithi019.stdout: "target_version": "0'0" 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "last_change": "22", 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend": "0", 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "last_force_op_resend_preluminous": "0", 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "auid": 0, 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_mode": "selfmanaged", 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_seq": 2, 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "snap_epoch": 22, 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "pool_snaps": [], 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps": "[]", 2024-09-15T21:35:03.209 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_bytes": 0, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "quota_max_objects": 0, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tiers": [], 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "tier_of": -1, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "read_tier": -1, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "write_tier": -1, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_mode": "none", 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_bytes": 0, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "target_max_objects": 0, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-15T21:35:03.210 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_flush_age": 0, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "cache_min_evict_age": 0, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profile": "", 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_params": { 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "none" 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_period": 0, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_count": 0, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "use_gmt_hitset": true, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min_read_recency_for_promote": 0, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "min_write_recency_for_promote": 0, 2024-09-15T21:35:03.211 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_grade_decay_rate": 0, 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "hit_set_search_last_n": 0, 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "grade_table": [], 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "stripe_width": 0, 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "expected_num_objects": 0, 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "fast_read": false, 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "options": {}, 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "application_metadata": { 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "rbd": {} 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.212 INFO:teuthology.orchestra.run.smithi019.stdout: "read_balance": { 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "score_type": "Fair distribution", 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "score_acting": 2.630000114440918, 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "score_stable": 2.630000114440918, 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "optimal_score": 1.1399999856948853, 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_acting": 3, 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "raw_score_stable": 3, 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity_weighted": 0.85879135131835938, 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity": 0.98000001907348633, 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: "average_primary_affinity_weighted": 0.86000001430511475 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.213 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "osds": [ 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "f7eb9035-e8c2-4d90-a198-2bed78d5a3b5", 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:35:03.214 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 66, 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6824", 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.215 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6825", 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.216 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6826", 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6827", 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.217 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6830", 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6831", 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.218 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6828", 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6829", 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 368705522 2024-09-15T21:35:03.219 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6825/368705522", 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6827/368705522", 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6831/368705522", 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6829/368705522", 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.220 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a", 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:35:03.221 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 66, 2024-09-15T21:35:03.222 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:35:03.222 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:35:03.224 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:35:03.224 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.224 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.224 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6816", 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6817", 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.225 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6818", 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6819", 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:35:03.226 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6822", 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.227 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6823", 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6820", 2024-09-15T21:35:03.228 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6821", 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3888640163 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6817/3888640163", 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6819/3888640163", 2024-09-15T21:35:03.229 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6823/3888640163", 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6821/3888640163", 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "180bae67-0f64-45a0-9a58-f4b8ff2b90a5", 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:35:03.230 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 36, 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.231 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6808", 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6809", 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:35:03.232 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6810", 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6811", 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.233 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6814", 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6815", 2024-09-15T21:35:03.234 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6812", 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.235 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6813", 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1831747042 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6809/1831747042", 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6811/1831747042", 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6815/1831747042", 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6813/1831747042", 2024-09-15T21:35:03.236 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "d05cec44-e228-4297-98c8-53e2ad91b37c", 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:35:03.237 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6800", 2024-09-15T21:35:03.238 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6801", 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.239 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6802", 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6803", 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.240 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6806", 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.241 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6807", 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6804", 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:35:03.242 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.19:6805", 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3658047863 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.19:6801/3658047863", 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.19:6803/3658047863", 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.19:6807/3658047863", 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.19:6805/3658047863", 2024-09-15T21:35:03.243 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "5d7097a2-8165-4885-a356-f8e67851be5d", 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:35:03.244 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 0.870330810546875, 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 0, 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.245 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6816", 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6817", 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.246 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6818", 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6819", 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.247 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6822", 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6823", 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:35:03.248 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6820", 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.249 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6821", 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1988554297 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6817/1988554297", 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6819/1988554297", 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6823/1988554297", 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6821/1988554297", 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:35:03.250 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f", 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:35:03.251 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 14, 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 52, 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6824", 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.252 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6825", 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.253 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6826", 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6827", 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:35:03.254 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6830", 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6831", 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.255 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6828", 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.256 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6829", 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1071217311 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6825/1071217311", 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6827/1071217311", 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6831/1071217311", 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6829/1071217311", 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:35:03.257 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612", 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 1, 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:35:03.258 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 13, 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 74, 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6800", 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.259 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6801", 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6802", 2024-09-15T21:35:03.260 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6803", 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.261 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6806", 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6807", 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.262 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6804", 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6805", 2024-09-15T21:35:03.263 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 1069316293 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6801/1069316293", 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6803/1069316293", 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6807/1069316293", 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6805/1069316293", 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:35:03.264 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "uuid": "fb77f072-c3ae-4914-81bd-473d30016320", 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "up": 1, 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "in": 1, 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "weight": 1, 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_affinity": 0, 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_begin": 0, 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "last_clean_end": 0, 2024-09-15T21:35:03.265 INFO:teuthology.orchestra.run.smithi019.stdout: "up_from": 16, 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: "up_thru": 43, 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: "down_at": 0, 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: "lost_at": 0, 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addrs": { 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6808", 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.266 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6809", 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addrs": { 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.267 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6810", 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6811", 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addrs": { 2024-09-15T21:35:03.268 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6814", 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6815", 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.269 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addrs": { 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: "addrvec": [ 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v2", 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6812", 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: "type": "v1", 2024-09-15T21:35:03.270 INFO:teuthology.orchestra.run.smithi019.stdout: "addr": "172.21.15.192:6813", 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "nonce": 3119270077 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "public_addr": "172.21.15.192:6809/3119270077", 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "cluster_addr": "172.21.15.192:6811/3119270077", 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_back_addr": "172.21.15.192:6815/3119270077", 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "heartbeat_front_addr": "172.21.15.192:6813/3119270077", 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "state": [ 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "exists", 2024-09-15T21:35:03.271 INFO:teuthology.orchestra.run.smithi019.stdout: "up" 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: "osd_xinfo": [ 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 0, 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:35:03.272 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.795411+0000", 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 1, 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:35:03.273 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.139237+0000", 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 2, 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.562583+0000", 2024-09-15T21:35:03.274 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 3, 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.363254+0000", 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:35:03.275 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 4, 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.344635+0000", 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.276 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 5, 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.323050+0000", 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 6, 2024-09-15T21:35:03.277 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:20.381780+0000", 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "osd": 7, 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "down_stamp": "0.000000", 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_probability": 0, 2024-09-15T21:35:03.278 INFO:teuthology.orchestra.run.smithi019.stdout: "laggy_interval": 0, 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: "features": 4540719139924082687, 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: "old_weight": 0, 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: "last_purged_snaps_scrub": "2024-09-15T21:31:21.049941+0000", 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: "dead_epoch": 0 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap": [ 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: { 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: "pgid": "2.5", 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: "osds": [ 2024-09-15T21:35:03.279 INFO:teuthology.orchestra.run.smithi019.stdout: 6, 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: 4 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: ] 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: ], 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_items": [], 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_upmap_primaries": [], 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: "pg_temp": [], 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: "primary_temp": [], 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: "blocklist": {}, 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: "range_blocklist": {}, 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: "erasure_code_profiles": { 2024-09-15T21:35:03.280 INFO:teuthology.orchestra.run.smithi019.stdout: "default": { 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "crush-failure-domain": "osd", 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "k": "2", 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "m": "1", 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "plugin": "jerasure", 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "technique": "reed_sol_van" 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: }, 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "removed_snaps_queue": [], 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "new_removed_snaps": [], 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "new_purged_snaps": [], 2024-09-15T21:35:03.281 INFO:teuthology.orchestra.run.smithi019.stdout: "crush_node_flags": {}, 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: "device_class_flags": {}, 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode": { 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_enabled": false, 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_bucket_count": 0, 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: "degraded_stretch_mode": 0, 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: "recovering_stretch_mode": 0, 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: "stretch_mode_bucket": 0 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout: } 2024-09-15T21:35:03.282 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:03.284 INFO:tasks.thrashosds.thrasher:j is {'epoch': 80, 'fsid': '13019d66-d4b6-4e2a-8322-38bcb4210bcf', 'created': '2024-09-15T21:31:12.329884+0000', 'modified': '2024-09-15T21:35:02.722933+0000', 'last_up_change': '2024-09-15T21:31:25.414443+0000', 'last_in_change': '2024-09-15T21:33:28.789828+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-15T21:31:22.622713+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.8587913513183594, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:31:30.313202+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 22, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.8587913513183594, 'average_primary_affinity': 0.9800000190734863, 'average_primary_affinity_weighted': 0.8600000143051147}}], 'osds': [{'osd': 0, 'uuid': 'f7eb9035-e8c2-4d90-a198-2bed78d5a3b5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6824', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6825', 'nonce': 368705522}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6826', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6827', 'nonce': 368705522}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6830', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6831', 'nonce': 368705522}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6828', 'nonce': 368705522}, {'type': 'v1', 'addr': '172.21.15.19:6829', 'nonce': 368705522}]}, 'public_addr': '172.21.15.19:6825/368705522', 'cluster_addr': '172.21.15.19:6827/368705522', 'heartbeat_back_addr': '172.21.15.19:6831/368705522', 'heartbeat_front_addr': '172.21.15.19:6829/368705522', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'd2bcecd4-0bb0-4b38-b395-8b6dbb48f53a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 66, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6816', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6817', 'nonce': 3888640163}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6818', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6819', 'nonce': 3888640163}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6822', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6823', 'nonce': 3888640163}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6820', 'nonce': 3888640163}, {'type': 'v1', 'addr': '172.21.15.19:6821', 'nonce': 3888640163}]}, 'public_addr': '172.21.15.19:6817/3888640163', 'cluster_addr': '172.21.15.19:6819/3888640163', 'heartbeat_back_addr': '172.21.15.19:6823/3888640163', 'heartbeat_front_addr': '172.21.15.19:6821/3888640163', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '180bae67-0f64-45a0-9a58-f4b8ff2b90a5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 36, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6808', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6809', 'nonce': 1831747042}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6810', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6811', 'nonce': 1831747042}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6814', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6815', 'nonce': 1831747042}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6812', 'nonce': 1831747042}, {'type': 'v1', 'addr': '172.21.15.19:6813', 'nonce': 1831747042}]}, 'public_addr': '172.21.15.19:6809/1831747042', 'cluster_addr': '172.21.15.19:6811/1831747042', 'heartbeat_back_addr': '172.21.15.19:6815/1831747042', 'heartbeat_front_addr': '172.21.15.19:6813/1831747042', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'd05cec44-e228-4297-98c8-53e2ad91b37c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6800', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6801', 'nonce': 3658047863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6802', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6803', 'nonce': 3658047863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6806', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6807', 'nonce': 3658047863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.19:6804', 'nonce': 3658047863}, {'type': 'v1', 'addr': '172.21.15.19:6805', 'nonce': 3658047863}]}, 'public_addr': '172.21.15.19:6801/3658047863', 'cluster_addr': '172.21.15.19:6803/3658047863', 'heartbeat_back_addr': '172.21.15.19:6807/3658047863', 'heartbeat_front_addr': '172.21.15.19:6805/3658047863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '5d7097a2-8165-4885-a356-f8e67851be5d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.870330810546875, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6816', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6817', 'nonce': 1988554297}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6818', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6819', 'nonce': 1988554297}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6822', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6823', 'nonce': 1988554297}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6820', 'nonce': 1988554297}, {'type': 'v1', 'addr': '172.21.15.192:6821', 'nonce': 1988554297}]}, 'public_addr': '172.21.15.192:6817/1988554297', 'cluster_addr': '172.21.15.192:6819/1988554297', 'heartbeat_back_addr': '172.21.15.192:6823/1988554297', 'heartbeat_front_addr': '172.21.15.192:6821/1988554297', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 52, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6824', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6825', 'nonce': 1071217311}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6826', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6827', 'nonce': 1071217311}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6830', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6831', 'nonce': 1071217311}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6828', 'nonce': 1071217311}, {'type': 'v1', 'addr': '172.21.15.192:6829', 'nonce': 1071217311}]}, 'public_addr': '172.21.15.192:6825/1071217311', 'cluster_addr': '172.21.15.192:6827/1071217311', 'heartbeat_back_addr': '172.21.15.192:6831/1071217311', 'heartbeat_front_addr': '172.21.15.192:6829/1071217311', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 74, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6800', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6801', 'nonce': 1069316293}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6802', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6803', 'nonce': 1069316293}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6806', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6807', 'nonce': 1069316293}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6804', 'nonce': 1069316293}, {'type': 'v1', 'addr': '172.21.15.192:6805', 'nonce': 1069316293}]}, 'public_addr': '172.21.15.192:6801/1069316293', 'cluster_addr': '172.21.15.192:6803/1069316293', 'heartbeat_back_addr': '172.21.15.192:6807/1069316293', 'heartbeat_front_addr': '172.21.15.192:6805/1069316293', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'fb77f072-c3ae-4914-81bd-473d30016320', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 16, 'up_thru': 43, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6808', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6809', 'nonce': 3119270077}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6810', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6811', 'nonce': 3119270077}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6814', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6815', 'nonce': 3119270077}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6812', 'nonce': 3119270077}, {'type': 'v1', 'addr': '172.21.15.192:6813', 'nonce': 3119270077}]}, 'public_addr': '172.21.15.192:6809/3119270077', 'cluster_addr': '172.21.15.192:6811/3119270077', 'heartbeat_back_addr': '172.21.15.192:6815/3119270077', 'heartbeat_front_addr': '172.21.15.192:6813/3119270077', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.795411+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.139237+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.562583+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.363254+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.344635+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.323050+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:20.381780+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540719139924082687, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-15T21:31:21.049941+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.5', 'osds': [6, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-15T21:35:03.285 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:35:03.285 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:03.285 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:03.286 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-15T21:35:03.434 INFO:teuthology.orchestra.run.smithi019.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:03.434 INFO:teuthology.orchestra.run.smithi019.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:03.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-15T21:35:03.559 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:03.559 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:03.568 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-15T21:35:03.622 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:03.622 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:35:03.634 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-15T21:35:02.727355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":528966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":223444,"kb_used_data":7220,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754751276,"statfs":{"total":773094113280,"available":772865306624,"internally_reserved":0,"allocated":7393280,"data_stored":4138922,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.082735"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":145,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-15T21:35:00.629706+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:35:00.629706+0000","last_peered":"2024-09-15T21:35:00.629706+0000","last_clean":"2024-09-15T21:35:00.629706+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:35:00.629706+0000","last_undegraded":"2024-09-15T21:35:00.629706+0000","last_fullsized":"2024-09-15T21:35:00.629706+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-15T21:35:00.107491+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:35:00.107491+0000","last_peered":"2024-09-15T21:35:00.107491+0000","last_clean":"2024-09-15T21:35:00.107491+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:35:00.107491+0000","last_undegraded":"2024-09-15T21:35:00.107491+0000","last_fullsized":"2024-09-15T21:35:00.107491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-15T21:35:00.629623+0000","last_change":"2024-09-15T21:34:49.389620+0000","last_active":"2024-09-15T21:35:00.629623+0000","last_peered":"2024-09-15T21:35:00.629623+0000","last_clean":"2024-09-15T21:35:00.629623+0000","last_became_active":"2024-09-15T21:34:49.389344+0000","last_became_peered":"2024-09-15T21:34:49.389344+0000","last_unstale":"2024-09-15T21:35:00.629623+0000","last_undegraded":"2024-09-15T21:35:00.629623+0000","last_fullsized":"2024-09-15T21:35:00.629623+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:07:42.104301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-15T21:35:00.107485+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:35:00.107485+0000","last_peered":"2024-09-15T21:35:00.107485+0000","last_clean":"2024-09-15T21:35:00.107485+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:35:00.107485+0000","last_undegraded":"2024-09-15T21:35:00.107485+0000","last_fullsized":"2024-09-15T21:35:00.107485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":97,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:34:57.129472+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:34:57.129472+0000","last_peered":"2024-09-15T21:34:57.129472+0000","last_clean":"2024-09-15T21:34:57.129472+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:34:57.129472+0000","last_undegraded":"2024-09-15T21:34:57.129472+0000","last_fullsized":"2024-09-15T21:34:57.129472+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":132,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-15T21:35:00.471181+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:35:00.471181+0000","last_peered":"2024-09-15T21:35:00.471181+0000","last_clean":"2024-09-15T21:35:00.471181+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:35:00.471181+0000","last_undegraded":"2024-09-15T21:35:00.471181+0000","last_fullsized":"2024-09-15T21:35:00.471181+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":75,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-15T21:35:00.107485+0000","last_change":"2024-09-15T21:34:32.071917+0000","last_active":"2024-09-15T21:35:00.107485+0000","last_peered":"2024-09-15T21:35:00.107485+0000","last_clean":"2024-09-15T21:35:00.107485+0000","last_became_active":"2024-09-15T21:34:32.071605+0000","last_became_peered":"2024-09-15T21:34:32.071605+0000","last_unstale":"2024-09-15T21:35:00.107485+0000","last_undegraded":"2024-09-15T21:35:00.107485+0000","last_fullsized":"2024-09-15T21:35:00.107485+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:40:46.837069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":87,"reported_epoch":78,"state":"active+clean","last_fresh":"2024-09-15T21:34:57.129492+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:34:57.129492+0000","last_peered":"2024-09-15T21:34:57.129492+0000","last_clean":"2024-09-15T21:34:57.129492+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:34:57.129492+0000","last_undegraded":"2024-09-15T21:34:57.129492+0000","last_fullsized":"2024-09-15T21:34:57.129492+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":177,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-15T21:35:00.629813+0000","last_change":"2024-09-15T21:34:32.072376+0000","last_active":"2024-09-15T21:35:00.629813+0000","last_peered":"2024-09-15T21:35:00.629813+0000","last_clean":"2024-09-15T21:35:00.629813+0000","last_became_active":"2024-09-15T21:34:32.072003+0000","last_became_peered":"2024-09-15T21:34:32.072003+0000","last_unstale":"2024-09-15T21:35:00.629813+0000","last_undegraded":"2024-09-15T21:35:00.629813+0000","last_fullsized":"2024-09-15T21:35:00.629813+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:03.086225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476783,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27952,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343888,"statfs":{"total":96636764160,"available":96608141312,"internally_reserved":0,"allocated":946176,"data_stored":911321,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476785,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27912,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343928,"statfs":{"total":96636764160,"available":96608182272,"internally_reserved":0,"allocated":905216,"data_stored":382393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476784,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343912,"statfs":{"total":96636764160,"available":96608165888,"internally_reserved":0,"allocated":921600,"data_stored":387883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476784,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27964,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343876,"statfs":{"total":96636764160,"available":96608129024,"internally_reserved":0,"allocated":958464,"data_stored":911321,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542191,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27916,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343924,"statfs":{"total":96636764160,"available":96608178176,"internally_reserved":0,"allocated":909312,"data_stored":382412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":387864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":387864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574896,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27924,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343916,"statfs":{"total":96636764160,"available":96608169984,"internally_reserved":0,"allocated":917504,"data_stored":387864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:35:03.635 INFO:tasks.thrashosds.thrasher:Setting 2.0 to [2, 4, 7, 5] 2024-09-15T21:35:03.636 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '2.0', '2', '4', '7', '5'] 2024-09-15T21:35:03.636 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 2.0 2 4 7 5 2024-09-15T21:35:03.672 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:03.672 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:03.681 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-15T21:35:03.785 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:03.785 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:03.794 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-15T21:35:03.896 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:03.896 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:03.905 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-15T21:35:04.007 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:04.008 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:04.016 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-15T21:35:04.120 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:04.120 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:04.128 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-15T21:35:04.130 INFO:teuthology.orchestra.run.smithi019.stderr:set 2.0 pg_upmap_items mapping to [2->4,7->5] 2024-09-15T21:35:04.232 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:04.232 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:04.240 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-15T21:35:04.344 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:04.344 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:04.353 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-15T21:35:04.456 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:04.456 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:04.465 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-15T21:35:04.568 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:04.568 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:04.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-15T21:35:04.680 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:04.681 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:04.689 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-15T21:35:04.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-15T21:35:04.791 INFO:teuthology.orchestra.run.smithi192.stdout:ERROR: (22) Invalid argument 2024-09-15T21:35:04.791 INFO:teuthology.orchestra.run.smithi192.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-15T21:35:05.218 INFO:teuthology.orchestra.run.smithi019.stderr:noscrub is unset 2024-09-15T21:35:05.232 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-15T21:35:06.220 INFO:teuthology.orchestra.run.smithi019.stderr:nodeep-scrub is unset 2024-09-15T21:35:09.149 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-15T21:35:09.280 INFO:teuthology.orchestra.run.smithi019.stdout:mon.a: {} 2024-09-15T21:35:09.280 INFO:teuthology.orchestra.run.smithi019.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:35:09.290 INFO:teuthology.orchestra.run.smithi019.stdout:mon.b: {} 2024-09-15T21:35:09.291 INFO:teuthology.orchestra.run.smithi019.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:35:09.299 INFO:teuthology.orchestra.run.smithi019.stdout:mon.c: {} 2024-09-15T21:35:09.299 INFO:teuthology.orchestra.run.smithi019.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-15T21:35:09.310 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-15T21:35:10.238 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.0 to 1 (10000) 2024-09-15T21:35:10.255 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-15T21:35:11.241 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-15T21:35:11.259 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-15T21:35:12.246 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.1 to 1 (10000) 2024-09-15T21:35:12.265 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-15T21:35:12.728 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-15T21:35:12.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-15T21:35:13.732 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.2 to 1 (10000) 2024-09-15T21:35:13.748 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-15T21:35:14.734 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-15T21:35:14.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-15T21:35:15.735 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.3 to 1 (10000) 2024-09-15T21:35:15.752 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-15T21:35:16.741 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-15T21:35:16.758 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-15T21:35:17.749 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.4 to 1 (10000) 2024-09-15T21:35:17.768 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-15T21:35:18.757 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-15T21:35:18.774 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-15T21:35:19.772 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.5 to 1 (10000) 2024-09-15T21:35:19.790 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-15T21:35:20.777 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-15T21:35:20.796 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-15T21:35:21.791 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.6 to 1 (10000) 2024-09-15T21:35:21.809 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-15T21:35:22.735 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-15T21:35:22.751 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-15T21:35:23.811 INFO:teuthology.orchestra.run.smithi019.stderr:reweighted osd.7 to 1 (10000) 2024-09-15T21:35:23.828 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-15T21:35:24.826 INFO:teuthology.orchestra.run.smithi019.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-15T21:35:24.842 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-15T21:35:24.842 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-15T21:35:24.843 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-15T21:35:24.843 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-15T21:35:24.843 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:35:24.843 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:35:25.150 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:25.150 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":100,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:35:24.810692+0000","last_up_change":"2024-09-15T21:31:25.414443+0000","last_in_change":"2024-09-15T21:33:28.789828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:31:22.622713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:31:30.313202+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6825","nonce":368705522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6827","nonce":368705522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6831","nonce":368705522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6829","nonce":368705522}]},"public_addr":"172.21.15.19:6825/368705522","cluster_addr":"172.21.15.19:6827/368705522","heartbeat_back_addr":"172.21.15.19:6831/368705522","heartbeat_front_addr":"172.21.15.19:6829/368705522","state":["exists","up"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6817","nonce":3888640163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6819","nonce":3888640163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6823","nonce":3888640163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6821","nonce":3888640163}]},"public_addr":"172.21.15.19:6817/3888640163","cluster_addr":"172.21.15.19:6819/3888640163","heartbeat_back_addr":"172.21.15.19:6823/3888640163","heartbeat_front_addr":"172.21.15.19:6821/3888640163","state":["exists","up"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6809","nonce":1831747042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6811","nonce":1831747042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6815","nonce":1831747042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6813","nonce":1831747042}]},"public_addr":"172.21.15.19:6809/1831747042","cluster_addr":"172.21.15.19:6811/1831747042","heartbeat_back_addr":"172.21.15.19:6815/1831747042","heartbeat_front_addr":"172.21.15.19:6813/1831747042","state":["exists","up"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6801","nonce":3658047863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6803","nonce":3658047863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6807","nonce":3658047863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6805","nonce":3658047863}]},"public_addr":"172.21.15.19:6801/3658047863","cluster_addr":"172.21.15.19:6803/3658047863","heartbeat_back_addr":"172.21.15.19:6807/3658047863","heartbeat_front_addr":"172.21.15.19:6805/3658047863","state":["exists","up"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6817","nonce":1988554297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6819","nonce":1988554297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6823","nonce":1988554297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6821","nonce":1988554297}]},"public_addr":"172.21.15.192:6817/1988554297","cluster_addr":"172.21.15.192:6819/1988554297","heartbeat_back_addr":"172.21.15.192:6823/1988554297","heartbeat_front_addr":"172.21.15.192:6821/1988554297","state":["exists","up"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6825","nonce":1071217311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6827","nonce":1071217311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6831","nonce":1071217311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6829","nonce":1071217311}]},"public_addr":"172.21.15.192:6825/1071217311","cluster_addr":"172.21.15.192:6827/1071217311","heartbeat_back_addr":"172.21.15.192:6831/1071217311","heartbeat_front_addr":"172.21.15.192:6829/1071217311","state":["exists","up"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6801","nonce":1069316293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6803","nonce":1069316293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6807","nonce":1069316293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6805","nonce":1069316293}]},"public_addr":"172.21.15.192:6801/1069316293","cluster_addr":"172.21.15.192:6803/1069316293","heartbeat_back_addr":"172.21.15.192:6807/1069316293","heartbeat_front_addr":"172.21.15.192:6805/1069316293","state":["exists","up"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6809","nonce":3119270077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6811","nonce":3119270077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6815","nonce":3119270077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6813","nonce":3119270077}]},"public_addr":"172.21.15.192:6809/3119270077","cluster_addr":"172.21.15.192:6811/3119270077","heartbeat_back_addr":"172.21.15.192:6815/3119270077","heartbeat_front_addr":"172.21.15.192:6813/3119270077","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.795411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.139237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.562583+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.363254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.344635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.323050+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.381780+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.049941+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[6,4]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:35:25.163 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:35:25.164 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:35:25.464 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:25.464 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":100,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:35:24.810692+0000","last_up_change":"2024-09-15T21:31:25.414443+0000","last_in_change":"2024-09-15T21:33:28.789828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:31:22.622713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:31:30.313202+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6825","nonce":368705522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6827","nonce":368705522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6831","nonce":368705522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6829","nonce":368705522}]},"public_addr":"172.21.15.19:6825/368705522","cluster_addr":"172.21.15.19:6827/368705522","heartbeat_back_addr":"172.21.15.19:6831/368705522","heartbeat_front_addr":"172.21.15.19:6829/368705522","state":["exists","up"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6817","nonce":3888640163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6819","nonce":3888640163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6823","nonce":3888640163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6821","nonce":3888640163}]},"public_addr":"172.21.15.19:6817/3888640163","cluster_addr":"172.21.15.19:6819/3888640163","heartbeat_back_addr":"172.21.15.19:6823/3888640163","heartbeat_front_addr":"172.21.15.19:6821/3888640163","state":["exists","up"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6809","nonce":1831747042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6811","nonce":1831747042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6815","nonce":1831747042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6813","nonce":1831747042}]},"public_addr":"172.21.15.19:6809/1831747042","cluster_addr":"172.21.15.19:6811/1831747042","heartbeat_back_addr":"172.21.15.19:6815/1831747042","heartbeat_front_addr":"172.21.15.19:6813/1831747042","state":["exists","up"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6801","nonce":3658047863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6803","nonce":3658047863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6807","nonce":3658047863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6805","nonce":3658047863}]},"public_addr":"172.21.15.19:6801/3658047863","cluster_addr":"172.21.15.19:6803/3658047863","heartbeat_back_addr":"172.21.15.19:6807/3658047863","heartbeat_front_addr":"172.21.15.19:6805/3658047863","state":["exists","up"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6817","nonce":1988554297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6819","nonce":1988554297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6823","nonce":1988554297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6821","nonce":1988554297}]},"public_addr":"172.21.15.192:6817/1988554297","cluster_addr":"172.21.15.192:6819/1988554297","heartbeat_back_addr":"172.21.15.192:6823/1988554297","heartbeat_front_addr":"172.21.15.192:6821/1988554297","state":["exists","up"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6825","nonce":1071217311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6827","nonce":1071217311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6831","nonce":1071217311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6829","nonce":1071217311}]},"public_addr":"172.21.15.192:6825/1071217311","cluster_addr":"172.21.15.192:6827/1071217311","heartbeat_back_addr":"172.21.15.192:6831/1071217311","heartbeat_front_addr":"172.21.15.192:6829/1071217311","state":["exists","up"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6801","nonce":1069316293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6803","nonce":1069316293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6807","nonce":1069316293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6805","nonce":1069316293}]},"public_addr":"172.21.15.192:6801/1069316293","cluster_addr":"172.21.15.192:6803/1069316293","heartbeat_back_addr":"172.21.15.192:6807/1069316293","heartbeat_front_addr":"172.21.15.192:6805/1069316293","state":["exists","up"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6809","nonce":3119270077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6811","nonce":3119270077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6815","nonce":3119270077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6813","nonce":3119270077}]},"public_addr":"172.21.15.192:6809/3119270077","cluster_addr":"172.21.15.192:6811/3119270077","heartbeat_back_addr":"172.21.15.192:6815/3119270077","heartbeat_front_addr":"172.21.15.192:6813/3119270077","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.795411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.139237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.562583+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.363254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.344635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.323050+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.381780+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.049941+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[6,4]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:35:25.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:35:25.478 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:35:25.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:35:25.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-15T21:35:25.479 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-15T21:35:25.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-15T21:35:25.480 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-15T21:35:25.481 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-15T21:35:25.734 INFO:teuthology.orchestra.run.smithi019.stdout:55834574901 2024-09-15T21:35:25.734 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:35:25.738 INFO:teuthology.orchestra.run.smithi019.stdout:55834574901 2024-09-15T21:35:25.738 INFO:teuthology.orchestra.run.smithi019.stdout:68719476792 2024-09-15T21:35:25.739 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:35:25.739 INFO:teuthology.orchestra.run.smithi019.stdout:60129542197 2024-09-15T21:35:25.740 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:35:25.741 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:35:25.741 INFO:teuthology.orchestra.run.smithi019.stdout:68719476789 2024-09-15T21:35:25.742 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:35:25.742 INFO:teuthology.orchestra.run.smithi019.stdout:68719476790 2024-09-15T21:35:25.743 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:35:25.743 INFO:teuthology.orchestra.run.smithi019.stdout:68719476790 2024-09-15T21:35:25.744 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:35:25.744 INFO:teuthology.orchestra.run.smithi019.stdout:55834574900 2024-09-15T21:35:25.745 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:35:26.324 INFO:teuthology.orchestra.run.smithi019.stdout:55834574900 2024-09-15T21:35:26.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574901 got 55834574900 for osd.2 2024-09-15T21:35:26.374 INFO:teuthology.orchestra.run.smithi019.stdout:55834574899 2024-09-15T21:35:26.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574899 for osd.6 2024-09-15T21:35:26.415 INFO:teuthology.orchestra.run.smithi019.stdout:55834574900 2024-09-15T21:35:26.417 INFO:teuthology.orchestra.run.smithi019.stdout:68719476788 2024-09-15T21:35:26.424 INFO:teuthology.orchestra.run.smithi019.stdout:68719476790 2024-09-15T21:35:26.427 INFO:teuthology.orchestra.run.smithi019.stdout:68719476788 2024-09-15T21:35:26.429 INFO:teuthology.orchestra.run.smithi019.stdout:60129542196 2024-09-15T21:35:26.433 INFO:teuthology.orchestra.run.smithi019.stdout:68719476787 2024-09-15T21:35:26.435 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476790 got 68719476788 for osd.0 2024-09-15T21:35:26.438 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574901 got 55834574900 for osd.3 2024-09-15T21:35:26.443 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476792 got 68719476790 for osd.4 2024-09-15T21:35:26.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476790 got 68719476788 for osd.1 2024-09-15T21:35:26.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542197 got 60129542196 for osd.5 2024-09-15T21:35:26.450 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476789 got 68719476787 for osd.7 2024-09-15T21:35:27.345 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:35:27.389 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-15T21:35:27.436 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:35:27.439 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-15T21:35:27.443 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-15T21:35:27.449 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:35:27.450 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-15T21:35:27.451 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-15T21:35:27.884 INFO:teuthology.orchestra.run.smithi019.stdout:55834574901 2024-09-15T21:35:27.901 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574901 got 55834574901 for osd.2 2024-09-15T21:35:27.901 DEBUG:teuthology.parallel:result is None 2024-09-15T21:35:27.967 INFO:teuthology.orchestra.run.smithi019.stdout:55834574900 2024-09-15T21:35:27.967 INFO:teuthology.orchestra.run.smithi019.stdout:68719476790 2024-09-15T21:35:27.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574900 got 55834574900 for osd.6 2024-09-15T21:35:27.989 DEBUG:teuthology.parallel:result is None 2024-09-15T21:35:27.989 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476790 got 68719476790 for osd.0 2024-09-15T21:35:27.989 DEBUG:teuthology.parallel:result is None 2024-09-15T21:35:28.027 INFO:teuthology.orchestra.run.smithi019.stdout:68719476792 2024-09-15T21:35:28.040 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476792 got 68719476792 for osd.4 2024-09-15T21:35:28.040 DEBUG:teuthology.parallel:result is None 2024-09-15T21:35:28.048 INFO:teuthology.orchestra.run.smithi019.stdout:68719476790 2024-09-15T21:35:28.048 INFO:teuthology.orchestra.run.smithi019.stdout:68719476789 2024-09-15T21:35:28.062 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476790 got 68719476790 for osd.1 2024-09-15T21:35:28.063 DEBUG:teuthology.parallel:result is None 2024-09-15T21:35:28.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476789 got 68719476789 for osd.7 2024-09-15T21:35:28.063 DEBUG:teuthology.parallel:result is None 2024-09-15T21:35:28.091 INFO:teuthology.orchestra.run.smithi019.stdout:60129542197 2024-09-15T21:35:28.096 INFO:teuthology.orchestra.run.smithi019.stdout:55834574901 2024-09-15T21:35:28.105 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542197 got 60129542197 for osd.5 2024-09-15T21:35:28.106 DEBUG:teuthology.parallel:result is None 2024-09-15T21:35:28.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574901 got 55834574901 for osd.3 2024-09-15T21:35:28.110 DEBUG:teuthology.parallel:result is None 2024-09-15T21:35:28.110 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-15T21:35:28.110 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:35:28.444 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:28.445 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:35:28.457 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-15T21:35:27.734871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":528966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225540,"kb_used_data":9296,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749180,"statfs":{"total":773094113280,"available":772863160320,"internally_reserved":0,"allocated":9519104,"data_stored":5091402,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.004281"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264892+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:35:25.264892+0000","last_peered":"2024-09-15T21:35:25.264892+0000","last_clean":"2024-09-15T21:35:25.264892+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:35:25.264892+0000","last_undegraded":"2024-09-15T21:35:25.264892+0000","last_fullsized":"2024-09-15T21:35:25.264892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:34:49.389620+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:34:49.389344+0000","last_became_peered":"2024-09-15T21:34:49.389344+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:07:42.104301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264896+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:35:25.264896+0000","last_peered":"2024-09-15T21:35:25.264896+0000","last_clean":"2024-09-15T21:35:25.264896+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:35:25.264896+0000","last_undegraded":"2024-09-15T21:35:25.264896+0000","last_fullsized":"2024-09-15T21:35:25.264896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":139,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264379+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:35:25.264379+0000","last_peered":"2024-09-15T21:35:25.264379+0000","last_clean":"2024-09-15T21:35:25.264379+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:35:25.264379+0000","last_undegraded":"2024-09-15T21:35:25.264379+0000","last_fullsized":"2024-09-15T21:35:25.264379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.169388+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:35:26.169388+0000","last_peered":"2024-09-15T21:35:26.169388+0000","last_clean":"2024-09-15T21:35:26.169388+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:35:26.169388+0000","last_undegraded":"2024-09-15T21:35:26.169388+0000","last_fullsized":"2024-09-15T21:35:26.169388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264370+0000","last_change":"2024-09-15T21:35:05.488995+0000","last_active":"2024-09-15T21:35:25.264370+0000","last_peered":"2024-09-15T21:35:25.264370+0000","last_clean":"2024-09-15T21:35:25.264370+0000","last_became_active":"2024-09-15T21:35:05.488798+0000","last_became_peered":"2024-09-15T21:35:05.488798+0000","last_unstale":"2024-09-15T21:35:25.264370+0000","last_undegraded":"2024-09-15T21:35:25.264370+0000","last_fullsized":"2024-09-15T21:35:25.264370+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:49:39.056773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":130,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264420+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:35:25.264420+0000","last_peered":"2024-09-15T21:35:25.264420+0000","last_clean":"2024-09-15T21:35:25.264420+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:35:25.264420+0000","last_undegraded":"2024-09-15T21:35:25.264420+0000","last_fullsized":"2024-09-15T21:35:25.264420+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-15T21:35:20.266492+0000","last_change":"2024-09-15T21:34:32.072376+0000","last_active":"2024-09-15T21:35:20.266492+0000","last_peered":"2024-09-15T21:35:20.266492+0000","last_clean":"2024-09-15T21:35:20.266492+0000","last_became_active":"2024-09-15T21:34:32.072003+0000","last_became_peered":"2024-09-15T21:34:32.072003+0000","last_unstale":"2024-09-15T21:35:20.266492+0000","last_undegraded":"2024-09-15T21:35:20.266492+0000","last_fullsized":"2024-09-15T21:35:20.266492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:03.086225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476789,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1204224,"data_stored":1026920,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476792,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":503500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476790,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1179648,"data_stored":503519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476790,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1228800,"data_stored":1032428,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542197,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28168,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343672,"statfs":{"total":96636764160,"available":96607920128,"internally_reserved":0,"allocated":1167360,"data_stored":498011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:35:28.458 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:35:28.756 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:28.757 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:35:28.769 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-15T21:35:28.650723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":528966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225572,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749148,"statfs":{"total":773094113280,"available":772863127552,"internally_reserved":0,"allocated":9543680,"data_stored":5102418,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.839732"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264892+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:35:25.264892+0000","last_peered":"2024-09-15T21:35:25.264892+0000","last_clean":"2024-09-15T21:35:25.264892+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:35:25.264892+0000","last_undegraded":"2024-09-15T21:35:25.264892+0000","last_fullsized":"2024-09-15T21:35:25.264892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:34:49.389620+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:34:49.389344+0000","last_became_peered":"2024-09-15T21:34:49.389344+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:07:42.104301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264896+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:35:25.264896+0000","last_peered":"2024-09-15T21:35:25.264896+0000","last_clean":"2024-09-15T21:35:25.264896+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:35:25.264896+0000","last_undegraded":"2024-09-15T21:35:25.264896+0000","last_fullsized":"2024-09-15T21:35:25.264896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":141,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168590+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:35:26.168590+0000","last_peered":"2024-09-15T21:35:26.168590+0000","last_clean":"2024-09-15T21:35:26.168590+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:35:26.168590+0000","last_undegraded":"2024-09-15T21:35:26.168590+0000","last_fullsized":"2024-09-15T21:35:26.168590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.169388+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:35:26.169388+0000","last_peered":"2024-09-15T21:35:26.169388+0000","last_clean":"2024-09-15T21:35:26.169388+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:35:26.169388+0000","last_undegraded":"2024-09-15T21:35:26.169388+0000","last_fullsized":"2024-09-15T21:35:26.169388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168589+0000","last_change":"2024-09-15T21:35:05.488995+0000","last_active":"2024-09-15T21:35:26.168589+0000","last_peered":"2024-09-15T21:35:26.168589+0000","last_clean":"2024-09-15T21:35:26.168589+0000","last_became_active":"2024-09-15T21:35:05.488798+0000","last_became_peered":"2024-09-15T21:35:05.488798+0000","last_unstale":"2024-09-15T21:35:26.168589+0000","last_undegraded":"2024-09-15T21:35:26.168589+0000","last_fullsized":"2024-09-15T21:35:26.168589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:49:39.056773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":132,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168589+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:35:26.168589+0000","last_peered":"2024-09-15T21:35:26.168589+0000","last_clean":"2024-09-15T21:35:26.168589+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:35:26.168589+0000","last_undegraded":"2024-09-15T21:35:26.168589+0000","last_fullsized":"2024-09-15T21:35:26.168589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-15T21:35:20.266492+0000","last_change":"2024-09-15T21:34:32.072376+0000","last_active":"2024-09-15T21:35:20.266492+0000","last_peered":"2024-09-15T21:35:20.266492+0000","last_clean":"2024-09-15T21:35:20.266492+0000","last_became_active":"2024-09-15T21:34:32.072003+0000","last_became_peered":"2024-09-15T21:34:32.072003+0000","last_unstale":"2024-09-15T21:35:20.266492+0000","last_undegraded":"2024-09-15T21:35:20.266492+0000","last_fullsized":"2024-09-15T21:35:20.266492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:03.086225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476789,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1204224,"data_stored":1026920,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476792,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":503500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476790,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1179648,"data_stored":503519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476790,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1228800,"data_stored":1032428,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1191936,"data_stored":509027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:35:28.770 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-15T21:35:28.770 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-15T21:35:29.097 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T21:35:29.107 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:35:29.107 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:35:29.402 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:29.402 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:35:29.414 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-15T21:35:28.650723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":528966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225572,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749148,"statfs":{"total":773094113280,"available":772863127552,"internally_reserved":0,"allocated":9543680,"data_stored":5102418,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.839732"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264892+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:35:25.264892+0000","last_peered":"2024-09-15T21:35:25.264892+0000","last_clean":"2024-09-15T21:35:25.264892+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:35:25.264892+0000","last_undegraded":"2024-09-15T21:35:25.264892+0000","last_fullsized":"2024-09-15T21:35:25.264892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:34:49.389620+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:34:49.389344+0000","last_became_peered":"2024-09-15T21:34:49.389344+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:07:42.104301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264896+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:35:25.264896+0000","last_peered":"2024-09-15T21:35:25.264896+0000","last_clean":"2024-09-15T21:35:25.264896+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:35:25.264896+0000","last_undegraded":"2024-09-15T21:35:25.264896+0000","last_fullsized":"2024-09-15T21:35:25.264896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":141,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168590+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:35:26.168590+0000","last_peered":"2024-09-15T21:35:26.168590+0000","last_clean":"2024-09-15T21:35:26.168590+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:35:26.168590+0000","last_undegraded":"2024-09-15T21:35:26.168590+0000","last_fullsized":"2024-09-15T21:35:26.168590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.169388+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:35:26.169388+0000","last_peered":"2024-09-15T21:35:26.169388+0000","last_clean":"2024-09-15T21:35:26.169388+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:35:26.169388+0000","last_undegraded":"2024-09-15T21:35:26.169388+0000","last_fullsized":"2024-09-15T21:35:26.169388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168589+0000","last_change":"2024-09-15T21:35:05.488995+0000","last_active":"2024-09-15T21:35:26.168589+0000","last_peered":"2024-09-15T21:35:26.168589+0000","last_clean":"2024-09-15T21:35:26.168589+0000","last_became_active":"2024-09-15T21:35:05.488798+0000","last_became_peered":"2024-09-15T21:35:05.488798+0000","last_unstale":"2024-09-15T21:35:26.168589+0000","last_undegraded":"2024-09-15T21:35:26.168589+0000","last_fullsized":"2024-09-15T21:35:26.168589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:49:39.056773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":132,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168589+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:35:26.168589+0000","last_peered":"2024-09-15T21:35:26.168589+0000","last_clean":"2024-09-15T21:35:26.168589+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:35:26.168589+0000","last_undegraded":"2024-09-15T21:35:26.168589+0000","last_fullsized":"2024-09-15T21:35:26.168589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-15T21:35:20.266492+0000","last_change":"2024-09-15T21:34:32.072376+0000","last_active":"2024-09-15T21:35:20.266492+0000","last_peered":"2024-09-15T21:35:20.266492+0000","last_clean":"2024-09-15T21:35:20.266492+0000","last_became_active":"2024-09-15T21:34:32.072003+0000","last_became_peered":"2024-09-15T21:34:32.072003+0000","last_unstale":"2024-09-15T21:35:20.266492+0000","last_undegraded":"2024-09-15T21:35:20.266492+0000","last_fullsized":"2024-09-15T21:35:20.266492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:03.086225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476789,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1204224,"data_stored":1026920,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476792,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":503500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476790,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1179648,"data_stored":503519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476790,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1228800,"data_stored":1032428,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1191936,"data_stored":509027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:35:29.415 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:35:29.712 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:29.713 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:35:29.725 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-15T21:35:28.650723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":528966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225572,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749148,"statfs":{"total":773094113280,"available":772863127552,"internally_reserved":0,"allocated":9543680,"data_stored":5102418,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.839732"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264892+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:35:25.264892+0000","last_peered":"2024-09-15T21:35:25.264892+0000","last_clean":"2024-09-15T21:35:25.264892+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:35:25.264892+0000","last_undegraded":"2024-09-15T21:35:25.264892+0000","last_fullsized":"2024-09-15T21:35:25.264892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:34:49.389620+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:34:49.389344+0000","last_became_peered":"2024-09-15T21:34:49.389344+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:07:42.104301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264896+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:35:25.264896+0000","last_peered":"2024-09-15T21:35:25.264896+0000","last_clean":"2024-09-15T21:35:25.264896+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:35:25.264896+0000","last_undegraded":"2024-09-15T21:35:25.264896+0000","last_fullsized":"2024-09-15T21:35:25.264896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":141,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168590+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:35:26.168590+0000","last_peered":"2024-09-15T21:35:26.168590+0000","last_clean":"2024-09-15T21:35:26.168590+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:35:26.168590+0000","last_undegraded":"2024-09-15T21:35:26.168590+0000","last_fullsized":"2024-09-15T21:35:26.168590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.169388+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:35:26.169388+0000","last_peered":"2024-09-15T21:35:26.169388+0000","last_clean":"2024-09-15T21:35:26.169388+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:35:26.169388+0000","last_undegraded":"2024-09-15T21:35:26.169388+0000","last_fullsized":"2024-09-15T21:35:26.169388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168589+0000","last_change":"2024-09-15T21:35:05.488995+0000","last_active":"2024-09-15T21:35:26.168589+0000","last_peered":"2024-09-15T21:35:26.168589+0000","last_clean":"2024-09-15T21:35:26.168589+0000","last_became_active":"2024-09-15T21:35:05.488798+0000","last_became_peered":"2024-09-15T21:35:05.488798+0000","last_unstale":"2024-09-15T21:35:26.168589+0000","last_undegraded":"2024-09-15T21:35:26.168589+0000","last_fullsized":"2024-09-15T21:35:26.168589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:49:39.056773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":132,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168589+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:35:26.168589+0000","last_peered":"2024-09-15T21:35:26.168589+0000","last_clean":"2024-09-15T21:35:26.168589+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:35:26.168589+0000","last_undegraded":"2024-09-15T21:35:26.168589+0000","last_fullsized":"2024-09-15T21:35:26.168589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-15T21:35:20.266492+0000","last_change":"2024-09-15T21:34:32.072376+0000","last_active":"2024-09-15T21:35:20.266492+0000","last_peered":"2024-09-15T21:35:20.266492+0000","last_clean":"2024-09-15T21:35:20.266492+0000","last_became_active":"2024-09-15T21:34:32.072003+0000","last_became_peered":"2024-09-15T21:34:32.072003+0000","last_unstale":"2024-09-15T21:35:20.266492+0000","last_undegraded":"2024-09-15T21:35:20.266492+0000","last_fullsized":"2024-09-15T21:35:20.266492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:03.086225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476789,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1204224,"data_stored":1026920,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476792,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":503500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476790,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1179648,"data_stored":503519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476790,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1228800,"data_stored":1032428,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1191936,"data_stored":509027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:35:29.726 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:35:29.726 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:35:30.023 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:30.023 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:35:30.036 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-15T21:35:28.650723+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":528966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225572,"kb_used_data":9320,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749148,"statfs":{"total":773094113280,"available":772863127552,"internally_reserved":0,"allocated":9543680,"data_stored":5102418,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.839732"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":186,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264892+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:35:25.264892+0000","last_peered":"2024-09-15T21:35:25.264892+0000","last_clean":"2024-09-15T21:35:25.264892+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:35:25.264892+0000","last_undegraded":"2024-09-15T21:35:25.264892+0000","last_fullsized":"2024-09-15T21:35:25.264892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.833984+0000","last_change":"2024-09-15T21:34:49.389620+0000","last_active":"2024-09-15T21:35:25.833984+0000","last_peered":"2024-09-15T21:35:25.833984+0000","last_clean":"2024-09-15T21:35:25.833984+0000","last_became_active":"2024-09-15T21:34:49.389344+0000","last_became_peered":"2024-09-15T21:34:49.389344+0000","last_unstale":"2024-09-15T21:35:25.833984+0000","last_undegraded":"2024-09-15T21:35:25.833984+0000","last_fullsized":"2024-09-15T21:35:25.833984+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:07:42.104301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":171,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-09-15T21:35:25.264896+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:35:25.264896+0000","last_peered":"2024-09-15T21:35:25.264896+0000","last_clean":"2024-09-15T21:35:25.264896+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:35:25.264896+0000","last_undegraded":"2024-09-15T21:35:25.264896+0000","last_fullsized":"2024-09-15T21:35:25.264896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":141,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168590+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:35:26.168590+0000","last_peered":"2024-09-15T21:35:26.168590+0000","last_clean":"2024-09-15T21:35:26.168590+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:35:26.168590+0000","last_undegraded":"2024-09-15T21:35:26.168590+0000","last_fullsized":"2024-09-15T21:35:26.168590+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":170,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.169388+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:35:26.169388+0000","last_peered":"2024-09-15T21:35:26.169388+0000","last_clean":"2024-09-15T21:35:26.169388+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:35:26.169388+0000","last_undegraded":"2024-09-15T21:35:26.169388+0000","last_fullsized":"2024-09-15T21:35:26.169388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168589+0000","last_change":"2024-09-15T21:35:05.488995+0000","last_active":"2024-09-15T21:35:26.168589+0000","last_peered":"2024-09-15T21:35:26.168589+0000","last_clean":"2024-09-15T21:35:26.168589+0000","last_became_active":"2024-09-15T21:35:05.488798+0000","last_became_peered":"2024-09-15T21:35:05.488798+0000","last_unstale":"2024-09-15T21:35:26.168589+0000","last_undegraded":"2024-09-15T21:35:26.168589+0000","last_fullsized":"2024-09-15T21:35:26.168589+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:49:39.056773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":132,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-15T21:35:26.168589+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:35:26.168589+0000","last_peered":"2024-09-15T21:35:26.168589+0000","last_clean":"2024-09-15T21:35:26.168589+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:35:26.168589+0000","last_undegraded":"2024-09-15T21:35:26.168589+0000","last_fullsized":"2024-09-15T21:35:26.168589+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":206,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-09-15T21:35:20.266492+0000","last_change":"2024-09-15T21:34:32.072376+0000","last_active":"2024-09-15T21:35:20.266492+0000","last_peered":"2024-09-15T21:35:20.266492+0000","last_clean":"2024-09-15T21:35:20.266492+0000","last_became_active":"2024-09-15T21:34:32.072003+0000","last_became_peered":"2024-09-15T21:34:32.072003+0000","last_unstale":"2024-09-15T21:35:20.266492+0000","last_undegraded":"2024-09-15T21:35:20.266492+0000","last_fullsized":"2024-09-15T21:35:20.266492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:03.086225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476789,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28208,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343632,"statfs":{"total":96636764160,"available":96607879168,"internally_reserved":0,"allocated":1204224,"data_stored":1026920,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476792,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28176,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343664,"statfs":{"total":96636764160,"available":96607911936,"internally_reserved":0,"allocated":1175552,"data_stored":503500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476790,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28184,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343656,"statfs":{"total":96636764160,"available":96607903744,"internally_reserved":0,"allocated":1179648,"data_stored":503519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476790,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1228800,"data_stored":1032428,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1191936,"data_stored":509027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574901,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28188,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343652,"statfs":{"total":96636764160,"available":96607899648,"internally_reserved":0,"allocated":1187840,"data_stored":509008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:35:30.037 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:35:30.336 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:30.336 INFO:teuthology.orchestra.run.smithi019.stdout:{"epoch":102,"fsid":"13019d66-d4b6-4e2a-8322-38bcb4210bcf","created":"2024-09-15T21:31:12.329884+0000","modified":"2024-09-15T21:35:27.729059+0000","last_up_change":"2024-09-15T21:31:25.414443+0000","last_in_change":"2024-09-15T21:33:28.789828+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-15T21:31:22.622713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-15T21:31:30.313202+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":22,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f7eb9035-e8c2-4d90-a198-2bed78d5a3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6824","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6825","nonce":368705522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6826","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6827","nonce":368705522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6830","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6831","nonce":368705522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6828","nonce":368705522},{"type":"v1","addr":"172.21.15.19:6829","nonce":368705522}]},"public_addr":"172.21.15.19:6825/368705522","cluster_addr":"172.21.15.19:6827/368705522","heartbeat_back_addr":"172.21.15.19:6831/368705522","heartbeat_front_addr":"172.21.15.19:6829/368705522","state":["exists","up"]},{"osd":1,"uuid":"d2bcecd4-0bb0-4b38-b395-8b6dbb48f53a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6816","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6817","nonce":3888640163}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6818","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6819","nonce":3888640163}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6822","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6823","nonce":3888640163}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6820","nonce":3888640163},{"type":"v1","addr":"172.21.15.19:6821","nonce":3888640163}]},"public_addr":"172.21.15.19:6817/3888640163","cluster_addr":"172.21.15.19:6819/3888640163","heartbeat_back_addr":"172.21.15.19:6823/3888640163","heartbeat_front_addr":"172.21.15.19:6821/3888640163","state":["exists","up"]},{"osd":2,"uuid":"180bae67-0f64-45a0-9a58-f4b8ff2b90a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6808","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6809","nonce":1831747042}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6810","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6811","nonce":1831747042}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6814","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6815","nonce":1831747042}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6812","nonce":1831747042},{"type":"v1","addr":"172.21.15.19:6813","nonce":1831747042}]},"public_addr":"172.21.15.19:6809/1831747042","cluster_addr":"172.21.15.19:6811/1831747042","heartbeat_back_addr":"172.21.15.19:6815/1831747042","heartbeat_front_addr":"172.21.15.19:6813/1831747042","state":["exists","up"]},{"osd":3,"uuid":"d05cec44-e228-4297-98c8-53e2ad91b37c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6800","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6801","nonce":3658047863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6802","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6803","nonce":3658047863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6806","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6807","nonce":3658047863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.19:6804","nonce":3658047863},{"type":"v1","addr":"172.21.15.19:6805","nonce":3658047863}]},"public_addr":"172.21.15.19:6801/3658047863","cluster_addr":"172.21.15.19:6803/3658047863","heartbeat_back_addr":"172.21.15.19:6807/3658047863","heartbeat_front_addr":"172.21.15.19:6805/3658047863","state":["exists","up"]},{"osd":4,"uuid":"5d7097a2-8165-4885-a356-f8e67851be5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6817","nonce":1988554297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6819","nonce":1988554297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6823","nonce":1988554297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":1988554297},{"type":"v1","addr":"172.21.15.192:6821","nonce":1988554297}]},"public_addr":"172.21.15.192:6817/1988554297","cluster_addr":"172.21.15.192:6819/1988554297","heartbeat_back_addr":"172.21.15.192:6823/1988554297","heartbeat_front_addr":"172.21.15.192:6821/1988554297","state":["exists","up"]},{"osd":5,"uuid":"b7e53438-0fa6-429a-9dd4-cc0b5dee5d6f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6825","nonce":1071217311}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6827","nonce":1071217311}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6831","nonce":1071217311}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":1071217311},{"type":"v1","addr":"172.21.15.192:6829","nonce":1071217311}]},"public_addr":"172.21.15.192:6825/1071217311","cluster_addr":"172.21.15.192:6827/1071217311","heartbeat_back_addr":"172.21.15.192:6831/1071217311","heartbeat_front_addr":"172.21.15.192:6829/1071217311","state":["exists","up"]},{"osd":6,"uuid":"14f40070-9e7d-4aa4-9cd1-7f9cf5fb1612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6801","nonce":1069316293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6803","nonce":1069316293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6807","nonce":1069316293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":1069316293},{"type":"v1","addr":"172.21.15.192:6805","nonce":1069316293}]},"public_addr":"172.21.15.192:6801/1069316293","cluster_addr":"172.21.15.192:6803/1069316293","heartbeat_back_addr":"172.21.15.192:6807/1069316293","heartbeat_front_addr":"172.21.15.192:6805/1069316293","state":["exists","up"]},{"osd":7,"uuid":"fb77f072-c3ae-4914-81bd-473d30016320","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6809","nonce":3119270077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6811","nonce":3119270077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6815","nonce":3119270077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":3119270077},{"type":"v1","addr":"172.21.15.192:6813","nonce":3119270077}]},"public_addr":"172.21.15.192:6809/3119270077","cluster_addr":"172.21.15.192:6811/3119270077","heartbeat_back_addr":"172.21.15.192:6815/3119270077","heartbeat_front_addr":"172.21.15.192:6813/3119270077","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.795411+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.139237+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.562583+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.363254+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.344635+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.323050+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:20.381780+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:31:21.049941+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"2.5","osds":[6,4]}],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":7,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:35:31.350 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T21:35:31.350 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:35:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:31.492 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:35:31.492 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:31.503 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T21:35:31.801 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T21:35:31.814 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T21:35:31.814 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:35:31.941 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:31.941 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:35:31.941 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:31.952 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T21:35:32.253 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T21:35:32.266 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T21:35:32.267 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:35:32.393 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:32.393 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:35:32.393 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:32.404 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T21:35:32.704 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T21:35:32.717 INFO:tasks.ceph:Scrubbing osd.3 2024-09-15T21:35:32.717 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:35:32.850 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:32.850 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:35:32.850 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:32.860 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-15T21:35:33.164 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 3 to deep-scrub 2024-09-15T21:35:33.177 INFO:tasks.ceph:Scrubbing osd.4 2024-09-15T21:35:33.178 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:35:33.305 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:33.305 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:35:33.305 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:33.316 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-15T21:35:33.615 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 4 to deep-scrub 2024-09-15T21:35:33.628 INFO:tasks.ceph:Scrubbing osd.5 2024-09-15T21:35:33.628 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:35:33.756 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:33.756 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:35:33.756 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:33.767 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-15T21:35:34.070 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 5 to deep-scrub 2024-09-15T21:35:34.083 INFO:tasks.ceph:Scrubbing osd.6 2024-09-15T21:35:34.083 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:35:34.210 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:34.211 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:35:34.211 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:34.221 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-15T21:35:34.519 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 6 to deep-scrub 2024-09-15T21:35:34.532 INFO:tasks.ceph:Scrubbing osd.7 2024-09-15T21:35:34.532 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:35:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:{ 2024-09-15T21:35:34.660 INFO:teuthology.orchestra.run.smithi019.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:35:34.660 INFO:teuthology.orchestra.run.smithi019.stdout:} 2024-09-15T21:35:34.670 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-15T21:35:34.970 INFO:teuthology.orchestra.run.smithi019.stderr:instructed osd(s) 7 to deep-scrub 2024-09-15T21:35:34.983 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:35:35.281 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:35.282 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:35:35.293 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-15T21:35:34.651834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225712,"kb_used_data":9476,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749008,"statfs":{"total":773094113280,"available":772862984192,"internally_reserved":0,"allocated":9703424,"data_stored":5173958,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.819126"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:29.074751+0000","last_change":"2024-09-15T21:33:29.805557+0000","last_active":"2024-09-15T21:35:29.074751+0000","last_peered":"2024-09-15T21:35:29.074751+0000","last_clean":"2024-09-15T21:35:29.074751+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:35:29.074751+0000","last_undegraded":"2024-09-15T21:35:29.074751+0000","last_fullsized":"2024-09-15T21:35:29.074751+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:21:27.477444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":175,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:29.074262+0000","last_change":"2024-09-15T21:31:33.465635+0000","last_active":"2024-09-15T21:35:29.074262+0000","last_peered":"2024-09-15T21:35:29.074262+0000","last_clean":"2024-09-15T21:35:29.074262+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:35:29.074262+0000","last_undegraded":"2024-09-15T21:35:29.074262+0000","last_fullsized":"2024-09-15T21:35:29.074262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:01.793919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":62,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:29.074728+0000","last_change":"2024-09-15T21:34:49.389620+0000","last_active":"2024-09-15T21:35:29.074728+0000","last_peered":"2024-09-15T21:35:29.074728+0000","last_clean":"2024-09-15T21:35:29.074728+0000","last_became_active":"2024-09-15T21:34:49.389344+0000","last_became_peered":"2024-09-15T21:34:49.389344+0000","last_unstale":"2024-09-15T21:35:29.074728+0000","last_undegraded":"2024-09-15T21:35:29.074728+0000","last_fullsized":"2024-09-15T21:35:29.074728+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:07:42.104301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":175,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:29.074252+0000","last_change":"2024-09-15T21:31:33.465724+0000","last_active":"2024-09-15T21:35:29.074252+0000","last_peered":"2024-09-15T21:35:29.074252+0000","last_clean":"2024-09-15T21:35:29.074252+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:35:29.074252+0000","last_undegraded":"2024-09-15T21:35:29.074252+0000","last_fullsized":"2024-09-15T21:35:29.074252+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:11:12.917256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"22'2","reported_seq":143,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:28.891656+0000","last_change":"2024-09-15T21:33:52.983175+0000","last_active":"2024-09-15T21:35:28.891656+0000","last_peered":"2024-09-15T21:35:28.891656+0000","last_clean":"2024-09-15T21:35:28.891656+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:35:28.891656+0000","last_undegraded":"2024-09-15T21:35:28.891656+0000","last_fullsized":"2024-09-15T21:35:28.891656+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:48:44.958827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":172,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:27.734728+0000","last_change":"2024-09-15T21:31:33.459367+0000","last_active":"2024-09-15T21:35:27.734728+0000","last_peered":"2024-09-15T21:35:27.734728+0000","last_clean":"2024-09-15T21:35:27.734728+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:35:27.734728+0000","last_undegraded":"2024-09-15T21:35:27.734728+0000","last_fullsized":"2024-09-15T21:35:27.734728+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:20:10.719603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":49,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:28.891647+0000","last_change":"2024-09-15T21:35:05.488995+0000","last_active":"2024-09-15T21:35:28.891647+0000","last_peered":"2024-09-15T21:35:28.891647+0000","last_clean":"2024-09-15T21:35:28.891647+0000","last_became_active":"2024-09-15T21:35:05.488798+0000","last_became_peered":"2024-09-15T21:35:05.488798+0000","last_unstale":"2024-09-15T21:35:28.891647+0000","last_undegraded":"2024-09-15T21:35:28.891647+0000","last_fullsized":"2024-09-15T21:35:28.891647+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:49:39.056773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"20'1","reported_seq":134,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:28.891649+0000","last_change":"2024-09-15T21:33:11.490545+0000","last_active":"2024-09-15T21:35:28.891649+0000","last_peered":"2024-09-15T21:35:28.891649+0000","last_clean":"2024-09-15T21:35:28.891649+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:35:28.891649+0000","last_undegraded":"2024-09-15T21:35:28.891649+0000","last_fullsized":"2024-09-15T21:35:28.891649+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:30.435089+0000","last_clean_scrub_stamp":"2024-09-15T21:31:30.435089+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:09:42.212425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":71,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:28.094242+0000","last_change":"2024-09-15T21:35:25.838937+0000","last_active":"2024-09-15T21:35:28.094242+0000","last_peered":"2024-09-15T21:35:28.094242+0000","last_clean":"2024-09-15T21:35:28.094242+0000","last_became_active":"2024-09-15T21:35:25.838678+0000","last_became_peered":"2024-09-15T21:35:25.838678+0000","last_unstale":"2024-09-15T21:35:28.094242+0000","last_undegraded":"2024-09-15T21:35:28.094242+0000","last_fullsized":"2024-09-15T21:35:28.094242+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:31:22.627903+0000","last_clean_scrub_stamp":"2024-09-15T21:31:22.627903+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:25:43.667698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476790,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28244,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343596,"statfs":{"total":96636764160,"available":96607842304,"internally_reserved":0,"allocated":1241088,"data_stored":1043436,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476793,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1200128,"data_stored":514508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476791,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1204224,"data_stored":514527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476791,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28256,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343584,"statfs":{"total":96636764160,"available":96607830016,"internally_reserved":0,"allocated":1253376,"data_stored":1043436,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542199,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1204224,"data_stored":514527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1200128,"data_stored":514508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1200128,"data_stored":514508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28204,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343636,"statfs":{"total":96636764160,"available":96607883264,"internally_reserved":0,"allocated":1200128,"data_stored":514508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:35:35.297 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T21:31:30.435089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.297 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T21:31:30.435089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.297 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T21:31:30.435089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.297 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T21:31:30.435089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.298 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T21:31:30.435089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.298 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T21:31:30.435089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.298 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T21:31:30.435089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.298 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T21:31:30.435089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.298 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T21:31:22.627903+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=31, tm_sec=22, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=35, tm_sec=30, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:35:35.298 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T21:35:55.299 DEBUG:teuthology.orchestra.run.smithi019:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:35:55.610 INFO:teuthology.orchestra.run.smithi019.stdout: 2024-09-15T21:35:55.611 INFO:teuthology.orchestra.run.smithi019.stderr:dumped all 2024-09-15T21:35:55.623 INFO:teuthology.orchestra.run.smithi019.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-15T21:35:54.655825+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":1057894,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":225676,"kb_used_data":9476,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754749044,"statfs":{"total":773094113280,"available":772863021056,"internally_reserved":0,"allocated":9703424,"data_stored":5173958,"data_compressed":11846,"data_compressed_allocated":65536,"data_compressed_original":1036288,"omap_allocated":12752,"internal_metadata":221236784},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002381"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":197,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:36.143640+0000","last_change":"2024-09-15T21:35:36.143556+0000","last_active":"2024-09-15T21:35:36.143640+0000","last_peered":"2024-09-15T21:35:36.143640+0000","last_clean":"2024-09-15T21:35:36.143640+0000","last_became_active":"2024-09-15T21:33:29.805256+0000","last_became_peered":"2024-09-15T21:33:29.805256+0000","last_unstale":"2024-09-15T21:35:36.143640+0000","last_undegraded":"2024-09-15T21:35:36.143640+0000","last_fullsized":"2024-09-15T21:35:36.143640+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:35:36.143474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:35:36.143474+0000","last_clean_scrub_stamp":"2024-09-15T21:35:36.143474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:14:22.969490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000498436,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:32.462865+0000","last_change":"2024-09-15T21:35:32.462785+0000","last_active":"2024-09-15T21:35:32.462865+0000","last_peered":"2024-09-15T21:35:32.462865+0000","last_clean":"2024-09-15T21:35:32.462865+0000","last_became_active":"2024-09-15T21:31:31.454058+0000","last_became_peered":"2024-09-15T21:31:31.454058+0000","last_unstale":"2024-09-15T21:35:32.462865+0000","last_undegraded":"2024-09-15T21:35:32.462865+0000","last_fullsized":"2024-09-15T21:35:32.462865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:35:32.462718+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:35:32.462718+0000","last_clean_scrub_stamp":"2024-09-15T21:35:32.462718+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:34.267207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049180800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":70,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:35.116969+0000","last_change":"2024-09-15T21:35:35.116969+0000","last_active":"2024-09-15T21:35:35.116969+0000","last_peered":"2024-09-15T21:35:35.116969+0000","last_clean":"2024-09-15T21:35:35.116969+0000","last_became_active":"2024-09-15T21:34:49.389344+0000","last_became_peered":"2024-09-15T21:34:49.389344+0000","last_unstale":"2024-09-15T21:35:35.116969+0000","last_undegraded":"2024-09-15T21:35:35.116969+0000","last_fullsized":"2024-09-15T21:35:35.116969+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:35:35.116860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:35:35.116860+0000","last_clean_scrub_stamp":"2024-09-15T21:35:35.116860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:52:48.639667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":184,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:33.420789+0000","last_change":"2024-09-15T21:35:33.420740+0000","last_active":"2024-09-15T21:35:33.420789+0000","last_peered":"2024-09-15T21:35:33.420789+0000","last_clean":"2024-09-15T21:35:33.420789+0000","last_became_active":"2024-09-15T21:31:31.455670+0000","last_became_peered":"2024-09-15T21:31:31.455670+0000","last_unstale":"2024-09-15T21:35:33.420789+0000","last_undegraded":"2024-09-15T21:35:33.420789+0000","last_fullsized":"2024-09-15T21:35:33.420789+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:35:33.420675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:35:33.420675+0000","last_clean_scrub_stamp":"2024-09-15T21:35:33.420675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:37.047500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055676200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"102'3","reported_seq":152,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:35.790989+0000","last_change":"2024-09-15T21:35:35.790989+0000","last_active":"2024-09-15T21:35:35.790989+0000","last_peered":"2024-09-15T21:35:35.790989+0000","last_clean":"2024-09-15T21:35:35.790989+0000","last_became_active":"2024-09-15T21:33:52.980821+0000","last_became_peered":"2024-09-15T21:33:52.980821+0000","last_unstale":"2024-09-15T21:35:35.790989+0000","last_undegraded":"2024-09-15T21:35:35.790989+0000","last_fullsized":"2024-09-15T21:35:35.790989+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'3","last_scrub_stamp":"2024-09-15T21:35:35.790899+0000","last_deep_scrub":"102'3","last_deep_scrub_stamp":"2024-09-15T21:35:35.790899+0000","last_clean_scrub_stamp":"2024-09-15T21:35:35.790899+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:46:44.771955+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:33.396743+0000","last_change":"2024-09-15T21:35:33.396691+0000","last_active":"2024-09-15T21:35:33.396743+0000","last_peered":"2024-09-15T21:35:33.396743+0000","last_clean":"2024-09-15T21:35:33.396743+0000","last_became_active":"2024-09-15T21:31:31.454020+0000","last_became_peered":"2024-09-15T21:31:31.454020+0000","last_unstale":"2024-09-15T21:35:33.396743+0000","last_undegraded":"2024-09-15T21:35:33.396743+0000","last_fullsized":"2024-09-15T21:35:33.396743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:35:33.396624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:35:33.396624+0000","last_clean_scrub_stamp":"2024-09-15T21:35:33.396624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:15:34.678757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041735699999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":57,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:36.781185+0000","last_change":"2024-09-15T21:35:36.781185+0000","last_active":"2024-09-15T21:35:36.781185+0000","last_peered":"2024-09-15T21:35:36.781185+0000","last_clean":"2024-09-15T21:35:36.781185+0000","last_became_active":"2024-09-15T21:35:05.488798+0000","last_became_peered":"2024-09-15T21:35:05.488798+0000","last_unstale":"2024-09-15T21:35:36.781185+0000","last_undegraded":"2024-09-15T21:35:36.781185+0000","last_fullsized":"2024-09-15T21:35:36.781185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:35:36.781101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:35:36.781101+0000","last_clean_scrub_stamp":"2024-09-15T21:35:36.781101+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:20:51.121494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.3","version":"102'2","reported_seq":143,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:34.762341+0000","last_change":"2024-09-15T21:35:34.762341+0000","last_active":"2024-09-15T21:35:34.762341+0000","last_peered":"2024-09-15T21:35:34.762341+0000","last_clean":"2024-09-15T21:35:34.762341+0000","last_became_active":"2024-09-15T21:33:11.487830+0000","last_became_peered":"2024-09-15T21:33:11.487830+0000","last_unstale":"2024-09-15T21:35:34.762341+0000","last_undegraded":"2024-09-15T21:35:34.762341+0000","last_fullsized":"2024-09-15T21:35:34.762341+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'2","last_scrub_stamp":"2024-09-15T21:35:34.762250+0000","last_deep_scrub":"102'2","last_deep_scrub_stamp":"2024-09-15T21:35:34.762250+0000","last_clean_scrub_stamp":"2024-09-15T21:35:34.762250+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:45:54.684843+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"102'74","reported_seq":81,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-09-15T21:35:35.176029+0000","last_change":"2024-09-15T21:35:35.176029+0000","last_active":"2024-09-15T21:35:35.176029+0000","last_peered":"2024-09-15T21:35:35.176029+0000","last_clean":"2024-09-15T21:35:35.176029+0000","last_became_active":"2024-09-15T21:35:25.838678+0000","last_became_peered":"2024-09-15T21:35:25.838678+0000","last_unstale":"2024-09-15T21:35:35.176029+0000","last_undegraded":"2024-09-15T21:35:35.176029+0000","last_fullsized":"2024-09-15T21:35:35.176029+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'74","last_scrub_stamp":"2024-09-15T21:35:35.175924+0000","last_deep_scrub":"102'74","last_deep_scrub_stamp":"2024-09-15T21:35:35.175924+0000","last_clean_scrub_stamp":"2024-09-15T21:35:35.175924+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:54:50.749512+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":38,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135168,"data_stored":1586784,"data_compressed":18174,"data_compressed_allocated":98304,"data_compressed_original":1560576,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":16,"seq":68719476794,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1241088,"data_stored":1043436,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":16,"seq":68719476797,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1200128,"data_stored":514508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476795,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1204224,"data_stored":514527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476795,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28248,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343592,"statfs":{"total":96636764160,"available":96607838208,"internally_reserved":0,"allocated":1253376,"data_stored":1043436,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":14,"seq":60129542203,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1204224,"data_stored":514527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1200128,"data_stored":514508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28200,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343640,"statfs":{"total":96636764160,"available":96607887360,"internally_reserved":0,"allocated":1200128,"data_stored":514508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574907,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28196,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343644,"statfs":{"total":96636764160,"available":96607891456,"internally_reserved":0,"allocated":1200128,"data_stored":514508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":528928,"data_compressed":5518,"data_compressed_allocated":32768,"data_compressed_original":512000,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":528928,"data_compressed":6328,"data_compressed_allocated":32768,"data_compressed_original":524288,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:35:55.624 DEBUG:teuthology.orchestra.run.smithi019:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T21:35:55.942 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T21:35:55.942 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T21:35:55.942 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:35:55.942 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:55.982 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:35:55.983 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:35:55.983 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.066 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:35:56.067 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:35:56.067 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.118 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:35:56.118 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-15T21:35:56.118 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.154 INFO:tasks.ceph.osd.3:Stopped 2024-09-15T21:35:56.155 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-15T21:35:56.155 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.185 INFO:tasks.ceph.osd.4:Stopped 2024-09-15T21:35:56.185 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-15T21:35:56.185 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.253 INFO:tasks.ceph.osd.5:Stopped 2024-09-15T21:35:56.253 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-15T21:35:56.253 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.320 INFO:tasks.ceph.osd.6:Stopped 2024-09-15T21:35:56.320 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-15T21:35:56.321 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.403 INFO:tasks.ceph.osd.7:Stopped 2024-09-15T21:35:56.404 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T21:35:56.404 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-15T21:35:56.404 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.430 INFO:tasks.ceph.mgr.y:Stopped 2024-09-15T21:35:56.430 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T21:35:56.430 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.459 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T21:35:56.459 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T21:35:56.460 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:35:56.460 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.482 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:35:56.482 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:35:56.482 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.533 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:35:56.534 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:35:56.534 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:35:56.545 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:35:56.546 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T21:35:56.546 DEBUG:teuthology.orchestra.run.smithi019:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW OPS\)' | egrep -v 'slow request' | head -n 1 2024-09-15T21:35:56.612 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:35:56.613 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T21:35:56.791 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:35:56.791 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T21:35:56.882 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:35:56.882 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T21:35:56.939 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:35:56.939 DEBUG:teuthology.orchestra.run.smithi019:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-15T21:35:56.990 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:35:56.990 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-15T21:35:57.143 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:35:57.143 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-15T21:35:57.207 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:35:57.207 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-15T21:35:57.257 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:35:57.257 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-15T21:35:57.306 INFO:tasks.ceph:Archiving mon data... 2024-09-15T21:35:57.307 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/data/mon.a.tgz 2024-09-15T21:35:57.307 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-09-15T21:35:57.323 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.DsAL5UaxQq 2024-09-15T21:35:57.324 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.DsAL5UaxQq 2024-09-15T21:35:57.470 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.DsAL5UaxQq 2024-09-15T21:35:57.554 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.DsAL5UaxQq is 395KB 2024-09-15T21:35:57.651 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.DsAL5UaxQq 2024-09-15T21:35:57.666 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/data/mon.c.tgz 2024-09-15T21:35:57.666 DEBUG:teuthology.orchestra.run.smithi019:> mktemp 2024-09-15T21:35:57.720 INFO:teuthology.orchestra.run.smithi019.stdout:/tmp/tmp.95ZLWUiCcm 2024-09-15T21:35:57.721 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.95ZLWUiCcm 2024-09-15T21:35:57.854 DEBUG:teuthology.orchestra.run.smithi019:> sudo chmod 0666 /tmp/tmp.95ZLWUiCcm 2024-09-15T21:35:57.939 DEBUG:teuthology.orchestra.remote:smithi019:/tmp/tmp.95ZLWUiCcm is 395KB 2024-09-15T21:35:58.007 DEBUG:teuthology.orchestra.run.smithi019:> rm -fr /tmp/tmp.95ZLWUiCcm 2024-09-15T21:35:58.023 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/data/mon.b.tgz 2024-09-15T21:35:58.023 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2024-09-15T21:35:58.041 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.8pK96dUHlJ 2024-09-15T21:35:58.042 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.8pK96dUHlJ 2024-09-15T21:35:58.186 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.8pK96dUHlJ 2024-09-15T21:35:58.270 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.8pK96dUHlJ is 395KB 2024-09-15T21:35:58.366 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.8pK96dUHlJ 2024-09-15T21:35:58.382 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T21:35:58.382 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:35:58.384 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:35:58.465 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:35:58.499 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:35:58.500 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T21:35:58.501 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/remote/smithi019/crash 2024-09-15T21:35:58.502 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:35:58.532 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/remote/smithi192/crash 2024-09-15T21:35:58.535 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:35:58.563 INFO:tasks.ceph:Compressing logs... 2024-09-15T21:35:58.563 DEBUG:teuthology.orchestra.run.smithi019:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:35:58.579 DEBUG:teuthology.orchestra.run.smithi192:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:35:58.602 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T21:35:58.646 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32978.log 2024-09-15T21:35:58.646 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32938.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.32978.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32978.log.gz 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.32938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32938.log.gz 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33098.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33189.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33098.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33098.log.gz 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33189.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33189.log.gz 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33058.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33149.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33058.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-15T21:35:58.647 INFO:teuthology.orchestra.run.smithi192.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33058.log.gz 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-client.admin.33149.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33149.log.gz 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33018.log 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.33018.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.33018.log.gz -5 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr: --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33229.log 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.33229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33229.log.gz 2024-09-15T21:35:58.648 INFO:teuthology.orchestra.run.smithi192.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:35:58.735 INFO:teuthology.orchestra.run.smithi192.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T21:35:58.756 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55612.log 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54477.log 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67158.log 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55612.log.gz 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60412.log 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54477.log.gz 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54372.log 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67158.log.gz 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60412.log.gz 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57999.log 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38502.log 2024-09-15T21:35:58.757 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54372.log.gz 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67452.log 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57999.log.gz 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:gzip/var/log/ceph/ceph-client.admin.38502.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.65100.log 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38502.log.gz 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67452.log.gz 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62736.log 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65100.log.gz 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42076.log 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62736.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62736.log.gz 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46622.log 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37963.log 2024-09-15T21:35:58.758 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42076.log.gz 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46622.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46622.log.gz 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50761.log 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47430.log 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37963.log.gz 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50761.log.gz 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65730.log 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60879.log 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47430.log.gz 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65310.log 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65730.log.gz 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67284.log 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60879.log.gz 2024-09-15T21:35:58.759 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47105.log 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65310.log.gz 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64801.log 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67284.log.gz 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59455.log 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47105.log.gz 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64801.log.gz 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67242.log 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59087.log 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59455.log.gz 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37956.log 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67242.log.gz 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54294.log 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59087.log.gz 2024-09-15T21:35:58.760 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66780.log 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37956.log.gz 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53508.log 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54294.log.gz 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39090.log 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66780.log.gz 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53508.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53508.log.gz 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65394.log 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39090.log: -- /var/log/ceph/ceph-client.admin.39516.log 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39090.log.gz 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41654.log 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65394.log.gz 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38301.log 2024-09-15T21:35:58.761 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39516.log.gz 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65907.log 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41654.log.gz 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55871.log 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38301.log.gz 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56070.log 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65907.log.gz 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44635.log 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55871.log.gz 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55448.log 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56070.log.gz 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44635.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44635.log.gz 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55448.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55448.log.gz 2024-09-15T21:35:58.762 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67326.log 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37624.log 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67326.log.gz 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54902.log 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37624.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37624.log.gz 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46761.log 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65911.log 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54902.log.gz 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46761.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46761.log.gz 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64843.log 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54234.log 2024-09-15T21:35:58.763 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65911.log.gz 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64843.log.gz 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52152.log 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64515.log 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54234.log.gz 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52152.log.gz 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63814.log 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62167.log 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64515.log.gz 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63814.log: 25.4% -- replaced with /var/log/ceph/ceph-client.admin.63814.log.gz 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59708.log 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55347.log 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62167.log.gz 2024-09-15T21:35:58.764 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58336.log 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59708.log.gz 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63516.log 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55347.log.gz 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58336.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58336.log.gz 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61984.log 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48174.log 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63516.log.gz 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61984.log.gz 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61441.log 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46345.log 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48174.log.gz 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61441.log.gz 2024-09-15T21:35:58.765 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38304.log 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66444.log 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46345.log.gz 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38304.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38304.log.gz 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51989.log 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43994.log 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66444.log.gz 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51989.log.gz 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66948.log 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49796.log 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43994.log.gz 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57650.log 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66948.log.gz 2024-09-15T21:35:58.766 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47956.log 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49796.log.gz 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57650.log.gz 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63078.log 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54984.log 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47956.log.gz 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63078.log.gz 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38306.log 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37368.log 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54984.log.gz 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49037.log 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38306.log.gz 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54641.log 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37368.log.gz 2024-09-15T21:35:58.767 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49037.log.gz 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39006.log 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46561.log 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54641.log.gz 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43972.log 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39006.log.gz 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46561.log.gz 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59048.log 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61157.log 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43972.log.gz 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37628.log 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59048.log.gz 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66864.log 2024-09-15T21:35:58.768 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61157.log.gz 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37628.log.gz 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38643.log 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62512.log 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66864.log.gz 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38643.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38643.log.gz 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60733.log 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67536.log 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62512.log.gz 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62407.log 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60733.log.gz 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57368.log 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67536.log.gz 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66245.log 2024-09-15T21:35:58.769 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62407.log.gz 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38838.log 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57368.log.gz 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39973.log 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66245.log.gz 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49215.log 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38838.log.gz 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51585.log 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39973.log.gz 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45614.log 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49215.log.gz 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37041.log 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51585.log.gz 2024-09-15T21:35:58.770 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67662.log 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45614.log.gz 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58943.log 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37041.log.gz 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67662.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.53125.log 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67662.log.gz 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61338.log 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58943.log.gz 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33762.log 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53125.log.gz 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49995.log 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61338.log.gz 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.33762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38166.log 2024-09-15T21:35:58.771 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33762.log.gz 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62573.log 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49995.log.gz 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65688.log 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38166.log.gz 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41394.log 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62573.log.gz 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58643.log 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65688.log.gz 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57031.log 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41394.log.gz 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58477.log 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58643.log.gz 2024-09-15T21:35:58.772 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40732.log 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57031.log.gz 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66242.log 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58477.log.gz 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64233.log 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40732.log.gz 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66242.log.gz 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46907.log 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64233.log.gz 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51729.log 0.0% 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45494.log 2024-09-15T21:35:58.773 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46907.log.gz 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63558.log 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51729.log.gz 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44732.log 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45494.log.gz 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60935.log 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63558.log.gz 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44732.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54032.log 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44732.log.gz 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60307.log 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60935.log.gz 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54373.log 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54032.log.gz 2024-09-15T21:35:58.774 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67620.log 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60307.log.gz 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51346.log 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54373.log.gz 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54922.log 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67620.log.gz 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40452.log 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51346.log.gz 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59820.log 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54922.log.gz 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67828.log 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40452.log.gz 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52840.log 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59820.log.gz 2024-09-15T21:35:58.775 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37623.log 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67828.log.gz 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65909.log 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52840.log.gz 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65604.log 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37623.log.gz 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53851.log 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65909.log.gz 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61922.log 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65604.log.gz 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53851.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53851.log.gz -5 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60306.log 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61983.log 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61922.log.gz 2024-09-15T21:35:58.776 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60306.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60306.log.gz 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65910.log 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61983.log.gz 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53605.log 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65910.log.gz 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65016.log 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66990.log 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53605.log.gz 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52678.log 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65016.log.gz 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61538.log 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66990.log.gz 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64056.log 2024-09-15T21:35:58.777 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52678.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52678.log.gz 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37969.log 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61538.log.gz 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33667.log 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64056.log.gz 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62406.log 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37969.log.gz 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.33667.log: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48114.log 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33667.log.gz 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58944.log 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62406.log.gz 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39703.log 2024-09-15T21:35:58.778 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48114.log.gz 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37452.log 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58944.log.gz 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65520.log 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39703.log.gz 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53871.log 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37452.log.gz 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47472.log 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65520.log.gz 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48317.log 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53871.log.gz 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47696.log 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47472.log.gz 2024-09-15T21:35:58.779 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59171.log 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48317.log.gz 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42035.log 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47696.log.gz 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39641.log 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59171.log.gz 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51386.log 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42035.log.gz 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67410.log 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39641.log.gz 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65226.log 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51386.log.gz 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67578.log 2024-09-15T21:35:58.780 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67410.log.gz 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39152.log 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65226.log.gz 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48854.log 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67578.log.gz 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52374.log 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39152.log.gz 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51220.log 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48854.log.gz 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56989.log 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52374.log.gz 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37634.log 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51220.log.gz 2024-09-15T21:35:58.781 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56650.log 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56989.log.gz 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38305.log 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37634.log.gz 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66241.log 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56650.log.gz 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41684.log 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38305.log.gz 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38641.log 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66241.log.gz 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60631.log 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41684.log.gz 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52415.log 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38641.log.gz 2024-09-15T21:35:58.782 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60631.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27541.log 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60631.log.gz 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65268.log 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52415.log.gz 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.tmp-client.admin.27541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39293.log 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27541.log.gz 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65184.log 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65268.log.gz 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60834.log 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39293.log.gz 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40233.log 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65184.log.gz 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65562.log 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60834.log.gz 2024-09-15T21:35:58.783 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58497.log 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40233.log.gz 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56772.log 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65562.log.gz 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58476.log 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58497.log.gz 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56772.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56772.log.gz 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37966.log 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58476.log.gz 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47514.log 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37966.log.gz 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63257.log 2024-09-15T21:35:58.784 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62168.log 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47514.log.gz 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52251.log 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63257.log.gz 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66246.log 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62168.log.gz 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60490.log 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52251.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52251.log.gz 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58316.log 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66246.log.gz 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60490.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.60490.log.gz 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64697.log 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40072.log 2024-09-15T21:35:58.785 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58316.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58316.log.gz 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61864.log 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64697.log.gz 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62635.log 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40072.log.gz 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52396.log 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61864.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61864.log.gz 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38964.log 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62635.log.gz 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55346.log 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52396.log.gz 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37627.log 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38964.log.gz 2024-09-15T21:35:58.786 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66247.log 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55346.log.gz 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47756.log 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37627.log.gz 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39869.log 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66247.log.gz 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37967.log 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47756.log.gz 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38299.log 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39869.log.gz 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57327.log/var/log/ceph/ceph-client.admin.37967.log: 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37967.log.gz 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39295.log 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38299.log.gz 2024-09-15T21:35:58.787 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44692.log 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57327.log.gz 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45114.log 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39295.log.gz 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47819.log 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44692.log.gz 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37630.log 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45114.log.gz 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60048.log 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47819.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47819.log.gz 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39294.log 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37630.log.gz 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60048.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60048.log.gz 2024-09-15T21:35:58.788 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61077.log 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39294.log.gz 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58785.log 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39292.log 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61077.log.gz 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44072.log 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58785.log.gz 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39292.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39292.log.gz 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53225.log 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44072.log.gz 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53386.log 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56330.log 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53225.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53225.log.gz 2024-09-15T21:35:58.789 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61539.log 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53386.log.gz 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43511.log 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56330.log.gz 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65436.log 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61539.log.gz 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.43511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44815.log 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43511.log.gz 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61076.log 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65436.log.gz 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65903.log 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44815.log.gz 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61076.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T21:35:58.790 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61076.log.gz 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67116.log 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65903.log.gz 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50519.log 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67116.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67116.log.gz 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67704.log 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50519.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66236.log 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39579.log 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67704.log.gz 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37970.log 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66236.log.gz 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43352.log 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39579.log.gz 2024-09-15T21:35:58.791 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42532.log 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37970.log: /var/log/ceph/ceph-client.admin.43352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46497.log 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37970.log.gz 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43352.log.gz 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42532.log.gz 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51828.log 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47045.log 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46497.log: /var/log/ceph/ceph-client.admin.51828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67368.log 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46497.log.gz 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51828.log.gz 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47045.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47045.log.gz 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67746.log 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.67368.log: /var/log/ceph/ceph-client.admin.58153.log 2024-09-15T21:35:58.792 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67368.log.gz 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67746.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67746.log.gz 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65142.log 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58153.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37968.log 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58153.log.gz 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61723.log 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65142.log.gz 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66248.log 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37968.log.gz 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64885.log 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61723.log.gz 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55165.log 2024-09-15T21:35:58.793 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66248.log: /var/log/ceph/ceph-client.admin.64885.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64885.log.gz 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.66248.log.gz 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55165.log.gz 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37275.log 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49655.log 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37275.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37275.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64969.log 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63394.log 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64969.log: /var/log/ceph/ceph-client.admin.49655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49655.log.gz 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64969.log.gz 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50256.log 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63394.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63394.log.gz 2024-09-15T21:35:58.794 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50256.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50256.log.gz 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66108.log 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38522.log 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66600.log 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66108.log.gz 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52939.log 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38522.log.gz/var/log/ceph/ceph-client.admin.66600.log: 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66600.log.gz 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38638.log 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52939.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52939.log.gz 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.55754.log 2024-09-15T21:35:58.795 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38638.log.gz 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38302.log 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55754.log.gz 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37250.log 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51583.log 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38302.log: /var/log/ceph/ceph-client.admin.37250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65646.log 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37250.log.gz 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38302.log.gz 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51583.log.gz 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59088.log 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65058.log 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65646.log.gz 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62470.log 2024-09-15T21:35:58.796 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59088.log.gz 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51770.log 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65058.log.gz 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62470.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62470.log.gz 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33365.log 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.51770.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51770.log.gz 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.52760.log 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.33365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33365.log.gz 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52780.log 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52760.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52760.log.gz 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55031.log 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52780.log.gz 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40410.log 2024-09-15T21:35:58.797 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55031.log.gz 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38637.log 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.40410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40410.log.gz 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37326.log 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45272.log 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38637.log.gz 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47269.log 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37326.log.gz 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45272.log: -- /var/log/ceph/ceph-client.admin.60128.log 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45272.log.gz 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47269.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63116.log 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47269.log.gz 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48435.log 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60128.log.gz 2024-09-15T21:35:58.798 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51264.log 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63116.log: gzip/var/log/ceph/ceph-client.admin.48435.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48435.log.gz 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr: /var/log/ceph/ceph-client.admin.39048.log 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0%/var/log/ceph/ceph-client.admin.51264.log: -- replaced with /var/log/ceph/ceph-client.admin.63116.log.gz 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51264.log.gz 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42315.log 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39048.log.gz 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38303.log 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39931.log 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42315.log.gz 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38642.log 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38303.log.gz 2024-09-15T21:35:58.799 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47268.log 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39931.log.gz 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44302.log 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38642.log: /var/log/ceph/ceph-client.admin.47268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44957.log 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47268.log.gz 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38642.log.gz 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44302.log.gz 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45174.log 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.44957.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44957.log.gz 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.64927.log 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45174.log.gz 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66906.log 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64927.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64927.log.gz 2024-09-15T21:35:58.800 INFO:teuthology.orchestra.run.smithi019.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48405.log 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66906.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66906.log.gz 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56200.log 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.48405.log: gzip -5 --verbose 27.7% -- /var/log/ceph/ceph-client.admin.59251.log 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48405.log.gz 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.56200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56200.log.gz 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55528.log 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.59251.log: -- /var/log/ceph/ceph-client.admin.64153.log 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59251.log.gz 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55528.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55528.log.gz 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55508.log 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64153.log.gz 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39457.log 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50096.log 2024-09-15T21:35:58.801 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55508.log: /var/log/ceph/ceph-client.admin.39457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65478.log 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39457.log.gz 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55508.log.gz 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.50096.log: 26.0% -- replaced with /var/log/ceph/ceph-client.admin.50096.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57680.log 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.65478.log: -- /var/log/ceph/ceph-client.admin.62937.log 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65478.log.gz 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.57680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57680.log.gz 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58396.log 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55973.log 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62937.log: /var/log/ceph/ceph-client.admin.58396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67200.log 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62937.log.gz 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58396.log.gz 2024-09-15T21:35:58.802 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67074.log 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.55973.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55973.log.gz 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59645.log 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67200.log: /var/log/ceph/ceph-client.admin.67074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37194.log 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67074.log.gz 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67200.log.gz 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59645.log.gz 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41762.log 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63917.log 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65906.log 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37194.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41762.log.gz 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37633.log 2024-09-15T21:35:58.803 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63917.log.gz 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50126.log 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65906.log.gz 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67032.log 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37633.log.gz 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.50126.log: -- /var/log/ceph/ceph-client.admin.66604.log 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50126.log.gz 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67032.log.gz 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47371.log 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66604.log.gz 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59271.log 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.47371.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53366.log 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47371.log.gz 2024-09-15T21:35:58.804 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53850.log 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59271.log.gz 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.53366.log: -- /var/log/ceph/ceph-client.admin.54539.log 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53366.log.gz 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53850.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45936.log 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53850.log.gz 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46866.log 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54539.log.gz 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54434.log 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.45936.log: /var/log/ceph/ceph-client.admin.46866.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41014.log 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45936.log.gz 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46866.log.gz 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.54434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54434.log.gz 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59862.log 2024-09-15T21:35:58.805 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.41014.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41014.log.gz 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49185.log 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59862.log.gz 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38644.log 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49185.log.gz 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42854.log 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37955.log 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38644.log.gz 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49937.log 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42854.log.gz 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.37955.log: -- /var/log/ceph/ceph-client.admin.59709.log 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37955.log.gz 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.49937.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49937.log.gz 2024-09-15T21:35:58.806 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65902.log 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64637.log 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59709.log: /var/log/ceph/ceph-client.admin.65902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52293.log 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59709.log.gz 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65902.log.gz 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64637.log.gz 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58663.log 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52293.log.gz 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66614.log 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.58663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58663.log.gz 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59904.log 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66614.log.gz 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38880.log 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59904.log.gz 2024-09-15T21:35:58.807 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66612.log 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38880.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38880.log.gz 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37410.log 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66602.log 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64378.log 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.37410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66612.log.gz 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37410.log.gz 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39807.log 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66602.log.gz 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.64378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66464.log 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64378.log.gz 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42472.log 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.39807.log: /var/log/ceph/ceph-client.admin.66464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62553.log 2024-09-15T21:35:58.808 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66464.log.gz 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39807.log.gz 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42472.log.gz 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38922.log 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59659.log 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.62553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62553.log.gz 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42153.log 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.38922.log: /var/log/ceph/ceph-client.admin.59659.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59659.log.gz 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38922.log.gz 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.42153.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42153.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67788.log 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59475.log 2024-09-15T21:35:58.809 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66494.log 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67788.log.gz 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.59475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59475.log.gz 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63754.log 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65912.log 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66494.log.gz 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.63754.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63754.log.gz 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52940.log 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46822.log 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.65912.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65912.log.gz 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.52940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52940.log.gz 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61400.log 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65352.log 2024-09-15T21:35:58.810 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.46822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46822.log.gz 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.61400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61400.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66822.log 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53789.log 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.66822.log: /var/log/ceph/ceph-client.admin.65352.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65352.log.gz 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66822.log.gz 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53324.log 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54740.log 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53789.log.gz 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.53324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53324.log.gz 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60371.log 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67494.log 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.60371.log: /var/log/ceph/ceph-client.admin.54740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60371.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54740.log.gz 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-15T21:35:58.811 INFO:teuthology.orchestra.run.smithi019.stderr:/var/log/ceph/ceph-client.admin.67494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67494.log.gz 2024-09-15T21:35:58.812 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T21:35:58.812 INFO:teuthology.orchestra.run.smithi192.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-15T21:35:58.824 INFO:teuthology.orchestra.run.smithi192.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T21:35:58.851 INFO:teuthology.orchestra.run.smithi192.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-15T21:35:58.886 INFO:teuthology.orchestra.run.smithi019.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T21:35:58.900 INFO:teuthology.orchestra.run.smithi019.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-15T21:35:58.971 INFO:teuthology.orchestra.run.smithi019.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T21:35:59.202 INFO:teuthology.orchestra.run.smithi192.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-15T21:35:59.221 INFO:teuthology.orchestra.run.smithi019.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T21:35:59.225 INFO:teuthology.orchestra.run.smithi192.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-15T21:35:59.227 INFO:teuthology.orchestra.run.smithi192.stderr: 2024-09-15T21:35:59.227 INFO:teuthology.orchestra.run.smithi192.stderr:real 0m0.604s 2024-09-15T21:35:59.227 INFO:teuthology.orchestra.run.smithi192.stderr:user 0m1.767s 2024-09-15T21:35:59.227 INFO:teuthology.orchestra.run.smithi192.stderr:sys 0m0.070s 2024-09-15T21:35:59.289 INFO:teuthology.orchestra.run.smithi019.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T21:35:59.291 INFO:teuthology.orchestra.run.smithi019.stderr: 2024-09-15T21:35:59.291 INFO:teuthology.orchestra.run.smithi019.stderr:real 0m0.699s 2024-09-15T21:35:59.291 INFO:teuthology.orchestra.run.smithi019.stderr:user 0m2.185s 2024-09-15T21:35:59.291 INFO:teuthology.orchestra.run.smithi019.stderr:sys 0m0.243s 2024-09-15T21:35:59.291 INFO:tasks.ceph:Archiving logs... 2024-09-15T21:35:59.292 DEBUG:teuthology.misc:Transferring archived files from smithi019:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/remote/smithi019/log 2024-09-15T21:35:59.293 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:35:59.909 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/remote/smithi192/log 2024-09-15T21:35:59.910 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:36:00.197 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T21:36:00.205 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T21:36:00.205 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:36:00.208 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:36:00.284 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T21:36:00.444 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T21:36:00.444 DEBUG:teuthology.orchestra.run.smithi019:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:36:00.447 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:36:00.461 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:36:00.462 INFO:teuthology.orchestra.run.smithi019.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:36:00.467 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:36:00.467 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2024-09-15T21:36:00.467 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 6 +1178us[+1178us] +/- 38ms 2024-09-15T21:36:00.467 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv02.front.sepia.ceph.com 3 6 377 8 -526us[ -238us] +/- 69ms 2024-09-15T21:36:00.467 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 -1591us[-1591us] +/- 74ms 2024-09-15T21:36:00.467 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:36:00.469 INFO:teuthology.orchestra.run.smithi019.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:36:00.469 INFO:teuthology.orchestra.run.smithi019.stdout:=============================================================================== 2024-09-15T21:36:00.469 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 5 +1173us[+1173us] +/- 38ms 2024-09-15T21:36:00.469 INFO:teuthology.orchestra.run.smithi019.stdout:^* hv02.front.sepia.ceph.com 3 6 377 6 -533us[ -248us] +/- 69ms 2024-09-15T21:36:00.469 INFO:teuthology.orchestra.run.smithi019.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 -1619us[-1333us] +/- 74ms 2024-09-15T21:36:00.469 INFO:teuthology.orchestra.run.smithi019.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:36:00.469 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T21:36:00.559 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T21:36:00.560 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T21:36:00.795 DEBUG:teuthology.orchestra.run.smithi019:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T21:36:00.890 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T21:36:00.969 DEBUG:teuthology.orchestra.run.smithi019:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T21:36:01.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:36:01.001 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T21:36:01.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:36:01.033 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T21:36:01.043 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T21:36:01.369 INFO:teuthology.task.internal:Duration was 698.456166 seconds 2024-09-15T21:36:01.370 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T21:36:01.413 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T21:36:01.413 DEBUG:teuthology.orchestra.run.smithi019:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:36:01.415 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:36:01.457 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:36:01.460 INFO:teuthology.orchestra.run.smithi019.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:36:01.902 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T21:36:01.903 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi019.front.sepia.ceph.com 2024-09-15T21:36:01.903 DEBUG:teuthology.orchestra.run.smithi019:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:36:01.927 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2024-09-15T21:36:01.927 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:36:01.956 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T21:36:01.957 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:36:01.969 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:36:02.026 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T21:36:02.026 DEBUG:teuthology.orchestra.run.smithi019:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:36:02.040 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:36:02.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T21:36:02.201 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T21:36:02.201 DEBUG:teuthology.orchestra.run.smithi019:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:36:02.227 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:36:02.252 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T21:36:02.313 DEBUG:teuthology.orchestra.run.smithi019:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:36:02.315 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:36:02.337 INFO:teuthology.orchestra.run.smithi019.stdout:kernel.core_pattern = core 2024-09-15T21:36:02.339 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2024-09-15T21:36:02.367 DEBUG:teuthology.orchestra.run.smithi019:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:36:02.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:36:02.422 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:36:02.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:36:02.436 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T21:36:02.446 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T21:36:02.447 DEBUG:teuthology.misc:Transferring archived files from smithi019:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/remote/smithi019 2024-09-15T21:36:02.447 DEBUG:teuthology.orchestra.run.smithi019:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:36:02.498 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905715/remote/smithi192 2024-09-15T21:36:02.498 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:36:02.532 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T21:36:02.532 DEBUG:teuthology.orchestra.run.smithi019:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:36:02.535 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:36:02.582 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T21:36:02.593 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T21:36:02.593 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T21:36:02.703 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T21:36:02.703 DEBUG:teuthology.orchestra.run.smithi019:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:36:02.706 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:36:02.719 INFO:teuthology.orchestra.run.smithi019.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:36 /home/ubuntu/cephtest 2024-09-15T21:36:02.721 INFO:teuthology.orchestra.run.smithi192.stdout: 266063 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:36 /home/ubuntu/cephtest 2024-09-15T21:36:02.722 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T21:36:02.755 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T21:36:02.988 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/classic msgr-failures/few msgr/async objectstore/{bluestore-options/write$/{write_v1} bluestore/bluestore-comp-zlib} rados tasks/mon_recovery validater/lockdep} duration: 698.4561657905579 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T21:36:02.989 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:36:03.062 INFO:teuthology.run:pass